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FOREWORD 


Professor  S,  S,  Wilks  outlined  the  form  of  the  firit  conference  on  the 
Design  of  Experiments  in  Army  Research,  Development  and  Testing. 
Basically,  this  meeting  was  composed  of  three  parts;  an  invited  speaker 
phatte  in  which  experts  were  invitod  to  discuss  recent  developments  and 
applications  in  the  field  of  design  of  experiment* ;  the  second  phase  con  ¬ 
sisted  of  technical  papers  presented  with  Army  personnel  discussing 
recent  successes  in  handling  their  problems,  and  finally  the  third  phase 
consisted  of  so-called  clinical  sessions.  The  speakers  for  these  sessions 
were  also  Army  personnel,  but  now  they  were  presenting  unsolved  problems. 
Nationally  known  scientists  served  as  panelists.  These  individuals,  as  well 
as  members  of  the  audience,  diacussed  the  unsolved  design  problems  and 
suggested  methods  of  attack  on  them.  Professor  Wilks  agreed  to  serve 
as  chairman  of  the  first  conference.  It  was  such  a  success  that  the  Army 
has  continued  the  meetings  on  a  yearly  basts.  Dr,  Wilks  served  as  their 
chairman  until  his  recent  untimely  death.  The  Army  Mathematics  Steering 
Committee,  sponsors  of  the  design  symposii.,  fully  realize  the  debt  they 
owe  to  Professor  Samuel  S.  Wilks  of  Princeton  University  and  have  asked 
that  the  next  conference,  the  tenth  in  this  series,  be  dedicated  to  him, 

At  the  Ninth  Conference  on  the  Design  of  Experiments,  Drs.  David 
Duncan,  Churchill  Eisenhart,  H,  0,  Hartley,  Solomon  Kullback  and  Frank 
Proschan  delivered  Invited  addresses.  Estimation  of  missile  trajectories, 
precision  and  accuracy,  non-linear  astimation,  communication  thuor  y, 
end  monotone  hazard  rates  in  systems  reliability  ware,  respectively,  the 
t'  ,<ics  treated  by  these  specialists.  Dr.  Craig  Crenshaw  was  invH.sd  by  ( 
the  host  to  be  the  Ait***  VJir.ner  Speaker.  Boyd  HarehbUigor  served  as 
the  Chairman  t-f  the  Panel  Discussion  on  wnat  types  of  Statisticians  are 
needed  in  Research  and  Development  Laboratories,  lie  nsked  ths  foll.ow- 
ihg  persons  to  serve  on  this  panel  and  discuss  various  aspects  of  tne 
topic;  E.  L.  Cox,  Churchill  Eieenhsrt,  Donald  Qardntr,  Frank  E,  Crntobs, 
John  L.  McDaniel,  Paul  1.  Rider,  and  William  Wolmen,  In  addition  to 
these  phases  of  the  program,  six  papers  were  givrn  in  the  Clinical 
Sessions  and  twenty-six  papers  in  the  Technical  Sessions. 

In  order  to  contribute  to  a  wide  dissemination  of  knowledge  and  use 
of  modern  statistical  principles  in  the  design  of  experiments,  the 
AMSCls  making  these  proceedings  available  to  all  interested  Army 
personnel.  The  present  volumn  contains  twenty-nine  of  the  papet  is  which 
were  presented  at  this  meeting, 
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Two  hundred  and  twenty -nine  registrant*  and  participants  from 
over  seventy  different  organisations  attended  the  Ninth  Conference. 
Speakers  and  panelists  came  from  the  Atlantic  Missile  Range;  Atomic 
Energy  Commission;  Biometrical  Services,  Agricultural  Research 
Services,  Plant  Industry  Station;  Boeing  Scientific  Research  Laboratories; 
Brigham  Young  University;  C-E-I.-R,  Inc,  ;  George  Washington  University; 
Goddard  Space  Plight  Center;  Johns  Hopkins  University;  NASA  Marshall 
Space  Flight  Center  (George  C.  Marshall  Space  Flight  Center);  National 
Bureau  of  Standards;  National  Institutes  of  Health;  North  Carolina 
State  University  of  the  University  of  North  Carolina  at  Raleigh; 

Oakridge  National  Laboratories;  Pan  Amorican  World  Airwaye; 

Patrick  Air  Force  Base;  Pratt  and  Whitney  Alrcrafti  Princeton; 

Research  Analysis  Corporation;  Research  Triangle  Institute;  Sandia 
Corporation;  Texas  A-M  University;  University  of  Wisconsin;  War 
Office,  United  Kingdom;  Wright  Patteraon  Air  Force  Baeei  and  ten 
Army  facilities. 

The  members  of  the  Army  Mathematic*  Steering  Committee  take 
this  opportunity  to  express  their  thanks  to  the  many  speakers  end 
other  research  workers  who  participated  in  the  conference;  to 
P>  igadier  General  John  Zierdt  for  having  the  U.  S.  Army  Missile 
Command  serve  as  host  and  for  making  available  the  faculties  under 
lue  command  at  Redstone  Arsenal  for  the  meetlngi  and  to  W,  H.  Ewart 
who  served  as  Chairman  on  Local  Arrangements.  Those  in  attendance 
are  indeed  indebted  to  him  for  the  excellent  handling  of  the  many  detail* 
nencled  to  make  euch  a  meeting  profitable  and  enjoyable.  Mr  Ewart 
was  ably  auolsted  in  these  talks  by  members  of  hie  committee;  E,  L. 
Bombara,  Eleanor  Colbert,  Hi"ry  A,  Dihm,  Don  Fulton,  Siegfried  H. 
Lel.nlgk,  and  Clyde  R.  Ward, 

A  complete  history  of  the  U.  S,  Army  Missile  Command  or  details 
of  t-hv  work  in  progress  at  Redstone  Arsenal  will  not  be  given  here,  Lut 
the  following  information  about  the  host  of  the  Ninth  Conference  on  the 
Design  of  Experiment*  should  prove  very  Interesting  to  those  in  atti.n.1- 
ance  These  remarks  end  the  following  picture  were  taken  from  the 
pamphlet  entitled  U,  S,  Army  Missiles  and  Rockets, 
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"Redstone  Arsenal,  Ala,  ,  is  the  home  of  U.  S,  Army  missiles  and 
rockets  where  a  dedicated  military-civilian  team  of  highly  trained 
specialists  see  to  it  that  American  soldiers  have  missile  and  rocket 
wespons  ready  for  use,  wherever  and  whenever  they  may  bn  needed 
in  the  defense  of  freedom. 

The  U,  S,  Army  Missile  Command,  which  is  a  major  commodity 
command  of  the  U,  S,  Army  Materiel  Command,  directs  worldwide 
Army  missile  activities  from  Its  headquarters  here.  Also  located 
at  Redstone  Arsenal  is  the  Army  Ordnance  Guided  Missile  School 
where  troops  of  the  U,  S,  Army,  the  Army'e  slater  services,  and 
allied  countries,  learn  to  be  missilemen.  The  Nation1*  top  priority 
program  of  perfecting  an  effective  defense  against  attack  by  Inter¬ 
continental  ballistic  missiles  is  directed  from  Redstone  Arsenal  by 
the  Nike  Zeus  Project  Office,  In  logistical  support  of  all  these  org¬ 
anizations  Is  the  Army  Missile  Support  Command,  which  Is  a  major 
element  of  the  Army  Missile  Command. 

U.  S.  Army  missiles  are  the  proud  products  of  the  people  who 
stand  behind  them  ••  more  than  12,  000  men  and  women  in  and  out 
of  uniform  who  are  engaged  in  this  vital  work  at  Redstone  Arsenal, 
They  are  responsible  for  managing,  developing  and  supporting  the 
lrroe  World's  most  versatile  missile  systems,  The  American 
soldier  depends  on  the  long  experience  and  know-how  of  these 
scientists,  technicians,  engineers,  secretaries  and  shopmen  to 
b'  'ng  a  missile  system  successfully  to  Ufa.  They  work  in  partner¬ 
ship  with  deienxc  contractor.*  across  the  nation  to  form  a  true  Army- 
Industry  team, 

It  takes  money  to  provide  up-to1  date  weapons  for  your  modern 
Army.  One  of  the  major  responsibilities  of  the  Army  missile  team 
i s;  to  make  sure  that  the  taxpayer  getr  a  fair  return  for  the  portion 
of  his  tax  dollar  allotted  Army  missiles.  This  means  a  billion 
dollar  a  yef.r  Investment  which  is  split  among  "niversitiea,  private 
research  institutions  and  American  industry  and  which  involves  rs,  ■■ 
than  10  prime  contractors,  300  first  tier  subcontractors  and  more 
than  3,  400  subcontractors  in  almost  every  state  in  the  union 
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Much  of  the  work  at  Redstone  U  done  behind  closed  doors  under 
tight  security  wre.pi.  In  the  Mlaeile  Command1!  eight  major  research 
and  development  laboratories,  tomorrow1!  micelle*  are  today's  work. 
The  Army  call*  it  "in-house"  reiearch.  It  ranges  acron  the  entire 
spectrum  of  missile  technology  including  such  actlvitlea  aa  experi¬ 
mentation  with  new  high  energy  rocket  fuels,  investigation  of  new 
meant  of  steering  missiles  and  advanced  work  in  metals,  chemicals, 
and  the  physical  sciences,  The  tools  of  the  trade  include  massive 
captive  test  stands  where  missiles  are  held  firmly  to  earth  as  their 
rocket  motors  roar,  lightning  fait  computers  analyzing  miloe  of 
test  data  in  their  electronic  brains,  and  ever  present  slide  rule  in 
the  hands  of  engineers,  The  single  word  facilities  is  used  to 
describe  land,  building#,  improvements  and  specialized  equipment. 
The  Army  investment  in  "facilities"  at  Redstone  is  approximately 
$300  million  dollars, 

Missile  hardware  is  the  end  product  of  our  resources  of  people, 
know-how,  money  and  facilities.  These  resources  combine  to  form 
the  Army  team  at  Redetone,  which  turns  out  superior  missiles  and 
their  supporting  equipment  that  do  a  variety  of  jobs  for  the  soldier. 

This  missile  hardware  covers  the  entire  spectrum  of  the  Soldier's 
needs  from  a  shoulder  fired,  two-pound,  tank  killer  to  the  two-etage 
Pershing  ballistic  missile  with  a  range  of  several  hundred  milea.  It 
includes  missiles  like  Sergeant,  a  quick  reacting,  solid  fuel  nuclear 
punch  for  the  field  Army,  and  the  Nike  Hercules  and  Hawk,  which 
g.'.ard  Army  unite  ov-est**-  and  American  cttlee  at  home  from  sir  ■ 
craft  attack. 


Their  shapes  may  differ  but  their  pedigree  is  the  same. 


THE  ARMY'S  MISSILE  FAMILY  -•  Here  stand*  ths  and 
products  of  ths  Army's  renearch,  development,  and 
tasting  in  missile  weapons,  Tha  larger  missiles,  left 
to  right,  are:  Nike  Hercules,  Sergeant,  Nike  Zeus, 
Pershing,  and  Nike  Ajax.  In  the  background  is  Lacrofcse. 
In  the  foreground  is  Hawk.  Soldiers  in  the  foreground 
hold,  left  to  right,  the  M-72  Rocket  Grenade,  the  Redeye, 
and  the  ENT  AC,  " 
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"COMPUTER  SIMULATION  STUDY  OF  BRUCETON  AND 
PROBIT  METHODS  OF  SENSITIVITY  TESTING11 
J,  B.  Cayie,  Propulsion  and  Vehicle  Engineering 
Divleton,  NA3A  Marshall  Space  Flight  Center 
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1330  CLINICAL  SESSION  A  Rocket  Auditorium 

Chairman:  Bertram  W,  Haines, 

Biometric*  Divleion,  Army  Biological 
Warfare  Laboratorlea,  Fort  Uetrick, 

Frederick,  Md. 

Panelists: 

Dr,  Waiter  D,  Foeter,  ilomathematica 
Division,  Army  Biologtcal  Warfare 
Laboratories 

Profeeeor  H,  O,  Hartley,  Initiate  of  Statistic  a, 

Agricultural  and  Mechanical  College  of  Texas 
Professor  H.  L.  Lucas,  Jr.,  Institute  of 
Statistics,  North  Carolina  State  College 
Dr,  Clifford  J.  Maloney,  Division  of  Biologies 
Standards,  National  Institute  of  Health 

"THE  IMPACT  OF  ADMINSTRAT1VE  LIMITATIONS 
ON  THE  DESIGN  OF  EXPERIMENTS" 

Gerhard  J,  Isaac,  Army  Medical  Research  and 
Nutrition  Laboratory,  Denvor,  Colorado 

"USE  OF  THE  M1TCHERLICH  EQUATION  IN  THE  THREE* 

PHASE  EXPERIMENTAL  DESIGN" 

Ardte  Lubln,  Dept,  of  Clinical  and  Social 
Psychology,  Neuropsychiatry  Div, ,  Walter  Reed 
Army  Institute  of  Research,  Walter  Reed  Army 
Medical  Center 

(1450-1530  COFFEE  BREaR  -  Lobby,  Rocket  Auditorium 

and  Cafeteria,  Bldg.  7101) 

1*5  <0  TECHNICAL  SESSION  III  Bldg,  71U 

Control  Room 


Chairman:  Alan  S.  Galbraith, 

Army  Reeearch  Office,  Durham,  N.  C. 

'! VERIFICATION  OF  PRODUCT  ACCEPTANCE 
INSPECTION  BY  VARIABLES" 

Henry  Ellner,  Directorate  for  Quality 
Assurance,  Edgewood  Arsenal,  Md. 


"VERIFICATION  OF  PRODUCT  ACCEPTANCE 
INSPECTION  BY  ATTRIBUTES" 

Joieph  M*ndcl*on,  Directorate  (or  Quality 
Aasurance,  Edgewood  Ariena1.  1/ld, 

TECHICAL  SESSION  IV  Bldg,  7101 

Confer ence  Room 


Chairman;  Frank  L,  Carr,  Army 
Material*  Raaaar.eh  Agency,  Watertown 
Anenal,  Watertown,  Maee. 

"ADDITIONAL  ANALYSIS  OF  MISSILE  TRAJECTORY 
MEASURING  SYSTEMS" 

Oliver  Lea  Kingsley  and  Bernle  R.  Froe, 

Range  Instrumentation  System*  Office, 

White  Sand*  Mieiile  Range,  New  Mexico 

"ASPECTS  TO  CONTROL  LIQUID  PROPELLANT 
SLOSHINO  BASED  UPON  EXISTINO  THEORY" 

Werner  R.  Built*,  PropuUion  and  Vehicle 
Engineering  Dlviaion,  NASA  Marahall  Space 
Flight  Center 

"BASIC  CONSIDERATIONS  FOR  THE  PRELIMINARY 
DESIGN  OF  A  SHOCK  TUBE  FOR  THE  EXPERIMENTAL 
INVESTIGATION  OF  THE  ACTION  OF  A  NUCLEAR  WAVE 
UPON  A  MISlilLE" 

Dietrich  E.  Gudaeiu,  Structure*  and  Mechanic* 
Laboratory,  Army  ivilseHe  Command 

CLINICAL  SESSION  B  Rocket  Auditorium 

Chairman;  Henry  A,  Dihm,  Advanced 
Sy*tem*  Laboratory,  Army  Missile  Command 

Panelist*; 

Professor  David  B.  Duncan,  John*  Hopkins  Unlv, 
and  Atlantic  MUnlle  Range 
Professor  Solomon  Rollback,  Dept,  cif 

StatUtlc*,  The  George  Wa»hington  Unlv. 

Dr.  Frank  Proschan,  Boeing  Scientific 
Reiearch  Laboratorle* 

Professor  O,  S,  Watson,  Dept,  of 

S.nUatlcii  John*  Hopkin*  University 


xitl 


"RELIABILITY  CONCEPTS  FOR  MISSILE  BATTERIES" 

Nicholas  Wilburn,  Army  Electronics  Research  and 
Development  Laboratory  Fort  Monmouth,  N,  J, 

"MONTE  CARLO  APPLICATION  FOR  DEVELOPING  A  DESIGN 
RELIABILITY  GOAL  COMPATIBLE  WITH  SMALL  SAMPLE 
REQUIREMENTS" 

Kay  Heathcock  and  Dais  L.  Burrows,  Propulsion  and 

Vehicle  Engineering  Division.  NASA  Marshall  Space  Flight  Center 

Thursday,  24  October  1963 

0830  TECHNICAL  SESSION  V  Bldg.  7101,  Confer,  Rm, 

Chairman!  Lt.  Colonel  Stefano  Vivona,  Division  of 
Communicable  Diseases,  Walter  Reed  Army  Institute 
of  Research 

"EXACT  MULTIPLE  CONTINGENCY  TABLE  ANALYSIS" 

Dorothy  Barg,  Morley  Leyton,  and  Clifford  Maloney, 

Division  of  Biologies  Standards,  National  Institute  of  Haalth 

"MICROSPECTROSCOPY  OF  TISSUES" 

Ooorge  1,  Lavin,  Army  Ballistics  Research  Laboratories, 
Aberdeen  Proving  Ground,  Md. 

(0930-1000  SHORT  BREAK) 

"RELATIONSHIP  OF  AEROSOL  STABILITY  TO  VIRULENCE  IN 
MICRO-ORGANISMS" 

IIvwuiuC.  Nlel*:n,  Dugway  Field  Office,  C-E-I-R,  INC.,  and 
Brigham  Young  "niv.  .  rep.  Dugway  Proving  Ground 

Minn  TECHNICAL  SESSION  VI  Rocket  Auditorium 

Chairman:  Badrig  Kurkjlan,  Harry  Diamond  Laboratories 

"DESIGN  FOR  THE  SEQUENTIAL  APPLICATION  OF  FACTORS" 

Sidney  Addelman,  Research  Triangle  Inst. ,  rep.  Army 
Research  Office,  Durham 

"2P  FACTORIAL  EXPERIMENTS  WITH  THE  FACTORS  APPLIED 
SEQUENTIALLY" 

R,  R.  Prairie  and  W.  J.  Zimmer,  Sandia  Corp, ,  rep, 

Atomic  Energy  Commieeion 
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(0950-1000  SHORT  BREAK) 

"ESTIMATION  OF  ERROR-SPECTRA  FROM  THE  CROSS¬ 
COVARIANCE  FUNCTIONS  OF  DIFFERENCES" 

D,  B,  Duncan,  Jchm  Kopkini.  Univ,  ,  and  W,  T. 

Welli,  Pan  American  World  Airwave,  Guided 
Mleille  Range  Dlv,  ,  PAFB,  Fla, 

0830  TECHNICAL  SESSION  VII  Bldg.  7101,  Control  Rm, 

Security  Cluaelflcationi  CONFIDENTIAL 

Chairman;  John  Purtell,  Watervliat 
Arienal,  Watervliat,  N,  Y. 

"A  COMPARISON  OF  DIFFERENT  METHODS  OF 
WEAPONS  EVALUATION" 

Andrew  J,  Ecklee,  III,  Research  Analyala  Corp, 

"DESIGN  OF  A  DATA  GATHERINQ  MODEL  FOR 
EVALUATING  SURFACE-TO-AIR  MISSILE 
SYSTEM  SUPPORT  REQUIREMENTS" 

Leon  Milter,  Reiearch  Analyiie  Corporation, 
presented  by  N.  Ray  Sumner 

(0950-1000  SHORT  BREAK) 

"AN  ANALYSIS  OF  HELICOPTER  RECONNAISSANCE 
TECHNIQUES" 

Arthur  P  .  Woode,  Research  Analytic  Corp. 

(1040-1120  COFFEE  BREAK  -  Lobby  of  Rocket  Auui um ) 

1120  GENERAL  SESSION  2  Rocket  Auditorium 

Chairman;  Dr,  Walter  D.  Foiter, 

Biometrici  Div, ,  Army  Biological 
Warfare  Labe,  Fort  Detrick,  Md, 

"REALISTIC  EVALUATION  OF  THE  PRECISION 
AND  ACCURACY  OF  INSTRUMENT  CALIBRATION 
SYSTEMS" 

Churchill  Eieenhart,  National  Bureau  of  Standard!, 

Waehington,  □.  C, 
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1230 

LUNCH 

Bldg. 

7101, 

Cafeteria 

1330 

TECHNICAL  SESSION  VIII 

Bldg. 

7101, 

Control  Rm, 

Security  Claeilflcktion:  CONFIDENTIAL 


Chairmen:  Harold  Faaaberg,  Research 
Analyei*  Corporation 

"STATISTICAL  STUDY  OF  RELIABILITY  AND 
ACCURACY  OF  SURFACE-TO-AIR  MISSILES" 

Bruce  Sterner,  Surveillance  Croup, 

Army  Ballletioa  Reeearch  Laboratc riee, 

Aberdeen  Proving  Ground,  Md. 

1330  TECHNICAL  SESSION  IX  Bldg.  7101,  Confer.  Rm, 

Chairman:  Miee  Aria  Weinert, 

Reeearch  Analyale  Corporation 

"NOTES  ON  FLEET  HOMOGENEITY  AND  HETEROGENEITY" 

G.  E.  Cooper,  Reeearch  Analyele  Corporation 

1330  TECHNICAL  SESSION  X  Rocket  Auditorium 

Chairman:  S,  L,  Bombara,  Propuleion 
and  Vehicle  Engineering  Divieion,  NASA 
Marehall  Space  Flight  Center 

"STATISTICAL  DECISION  THEORY" 

Lionel  Wwiee,  Army  Mathematic  *  Reeearch 
Gunter,  Uni  verify  of  Wieconein 

(1413-14&0  COFFEE  BREAK  •  Lobby  of  Rocket  Auditorium) 

14S0  GENERAL  SESSION  3  Rocket  Auditorium 

PANEL  DISCUSSION  ON  WHAT  TYPE  OF  STATISTICIANS 
ARE  NEEDED  IN  RESEARCH  AND  DEVELOPMENT 
LABORATORIES 

Chairman:  Proieeeor  Boyd  Harehbarger, 

Virginia  Polytechnic  Institute 


Panellist*: 

Dr.  E,  L,  Cox,  Biometrical  Service*,  ARS,  PI  ant 
Industry  Station 

Dr.  Churchill  Eltenhort,  National  Bureau  of  Standard* 

Dr.  Frank  E.  Grubbs,  Army  Ballistic*  Research 
laboratories,  Aberdeen,  Md, 

Mr.  John  L,  McDaniel,  Directorate  of  RfcD, 

Army  Missile  Command 

Dr.  Paul  R.  Rldsr,  Office  of  Aero-Space 
Research,  Wrlght-Patterson  AFB 

Dr.  William  Wolman,  Goddard  Space  Flight 
Center,  NASA 

Dr.  Donald  A.  Gardiner,  Mathematics  Dlv, , 

Oiikrtdg*  National  Laboratories 

1900  SOCIAL  HOUR  AND  DINNER  Redstone  Arsenal 

Officers'  Open  Mess 

Evening  Session  Chairman: 

Mr.  John  L,  McDaniel,  Technical 
Director  of  RAD,  Army  Missile  Command 

After  Dinner  Speaker: 

Dr.  Craig  M,  Crenshaw,  Chief  Scientist, 

U.  S.  Army  Materiel  Command 


25  October  1963 


0830  TECHNICAL  SESSION  XI  Rocket  Auditorium 

Chairman:  Erwin  Blser,  System  Division, 

Surveillance  Department,  Army  Electronics 
Research  and  Development  Laboratory, 

Fort  Monmouth,  N.  J. 


"AN  ANALYSIS  OF  FACTORIAL  EXPERIMENTAL  DESIONS 
L,  W.  Keeling,  Prapulelon  Laboratory,  Army  Mieslle 
Command 


"SOME  ASPECTS  OF  ANALYSIS  OF  PARTIALLY  FACTORIAL 
EXPERIMENTS" 

Scott  A,  Krane,  Dugway  Field  Office, 

C-E-I-R,  INC.  ,  representing  Dugway  Proving  Ground 


t 


0830 


0830 

t 


TECHNICAL  SESSION  XII 


xvil 

Bldg.  7101 
Control  Room 


Chairman!  Robert  E.  Welgle,  Watsrvliet, 

Arsenal,  Watervllet,  N,  Y, 

"RELIABILITY  ESTIMATION  FOR  MULTI- COMPONENT 
SYSTEMS" 

Jtmei  R.  Kniei,  Surveillance  Croup,  Army 
Ballistics  Research  Laboratories, 

Aberdeen  Proving  Ground,  Md, 

"STATISTICAL  STUDY  OF  AQINO  CHARACTERISTICS 
OF  ARTILLERY  MISSILES" 

Raymond  Bell,  Surveillance  Oroup,  Army 
Ballistic  Research  Laboratories, 

Aberdeen  Proving  Ground,  Md, 

CLINICAL  SESSION  C  Bldg.  7101 

Conference  Room 

The  paper  by  D,  H,  Chaddock  carries  a  security 
classification  of  CONFIDENTIAL:  the  paper  by 
Lester  Kate  is  not  olasslfisd. 

Chairman;  Fred  Frishman,  U.  S,  Army 
Research  Office,  Washington,  D.  C, 

Panelists; 

Dr,  O  P,  Bruno,  Surveillance  Oroup,  Army 
Ballistics  Research  Laboratories 
Dr,  F,  E,  Crubbs,  Army  Ballistics  Research 
Laboratories 

Dr,  Lionel  Weiss,  Army  Mathematics  Research 
Center,  University  of  Wisconsin 
Professor  S,  8,  Wilks,  Princeton  University 

"THE  MEASUREMENT  OF  THE  MORALE  AND  SUPPRESSIVE 
EFFECTS  OF  WEAPONS" 

D,  H,  Chaddock,  F,sq, ,  C,  B,  E. ,  Director  of  Artillsry 
Research  and  Development,  The  War  Offlca,  United 
Kingdom  (prenented  by  O,  F.  Komlosy,  presently  attached 
to  the  Research  Analysis  Corporation) 
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xvltl 


"THE  DESIGN  OF  AN  EXPERIMENTAL  PROGRAM  TO 
INVESTIGATE  BUCKLING  IN  THE  HOOP  DIRECTION 
OF  PR  ATI  ALLY  FILLED  BULKHEADS  UNDER  LOADING" 
Lester  Kats,  Propulsion  and  Vehicle  Engineering 
Division,  NASA  Marshall  Spaco  Flight  Center 

<09*0 -1020  COFFEE  BREAK  -  Lobby  of  Rocket  Auditorium) 

1020  GENERAL  SESSION  4  Rocket  Auditorium 

Chairman:  Profesor  S,  S.  Wilks, 

Princeton  University 

"NONLINEAR  ESTIMATION" 

H  O,  Hartley,  Institute  of  Statistics, 

Agricultural  and  Mechanical  College 
of  Texas 

"ON  THE  SIMULTANEOUS  ESTIMATION  OF  A  MISSILE 

TRAJECTORY  AND  THE  ERROR  VARIANCE  COMPONENTS 
INCLUDING  THE  ERROR  POWER  SPECTRA  OF  SEVERAL 
TRACKING  SYSTEMS" 

David  B,  Duncan,  Professor  of  Statistics  and  Bio- 
statistics,  Johns  Hopkins  Univsrsity,  and  Statistical 
Consultant,  Atlantic  Missile  Range 

12  30  LUNCH  Bldg.  7101, 

Cafeteria 

1  330-1530  TOUR  OF  ARSENAL  FACILITIES 


COMMUNICATION  THEORY 


Solomon  Kullhack 
Department  of  Statiitici 
The  Oeorge  Waihington  University 


This  presentation  will  cover  one  aspect  of  communication  theory  and 
I  shall  try  to  relate  some  of  the  concepts  of  interest  to  the  communication 
engineer  with  the  general  subject  of  thie  conference.  In  particular  1  shall 
talk  about  certain  facets  of  information  theory,  an  important  field  of  con¬ 
temporary  probability  theory  and  atatletics. 

I  shall  not  take  the  time  necessary  to  engage  in  a  detailed  discussion 
motivating  the  technical  definitions  of  measures  of  information  and  informa¬ 
tion  theory,  except  to  remark  that  among  the  various  definitions  which  have 
been  given  is  one  that  states  "information  is  a  measure  of  time  or  cost  of 
a  sort,  which  is  of  particular  use  to  the  engineer  in  his  role  of  designer 
of  an  experiment". 


Suppose  a  system  (or  information  source)  has  c  different  possible 
events  or  categories  or  messages  A^,  A ,  .  ,  ,  A^  with  respective 

probabilities  of  occurrence  p,  ,  p, . p^,  g  p.  -  1 , 

iel  1 


(  ^1  * 


1  he  expression 


H(0)  =  -Pjlog  pj  -  pzlog  p2  - 
*  E  (h) 


P  log  P,. 

r  c 


where  h  is  a  random  variable  which  takes  on  the  values  log  —  .  l^g  —  , 

1  pl 

.  .  .  ,  log  —  ,  is  called  the  entropy  of  the  system  (>,  by  analogy  with 

pc 

a  similar  concept  and  mathematical  expression  in  statistical  mechanics. 

The  entropy  is  interpreted  as  the  mean  uncertainty  about  g  prior  to  an 
observation,  or  the  mean  information  about  p  provided  by  an  observation. 

The  base  of  the  logarithm  is  r)uite  arbitrary  and  is  just  a  unit  of  measurement. 


2 


Design  of  Experiments 


The  feet  that 

(3)  n  »  b*  =  ay,  x  ®  logb  n,  y  *  logp  n,  logb  n  ■  logb  &  <  log*  n 

permits  ready  change  from  one  logarithmic  banc  to  another.  The  following 
Table  1  indicates  the  unit  in  which  H  (4>)  is  measured  for  the  more  common 
bases  of  logarithms 


Table  1 


Base 

Unit 

!  2 

bit 

1  • 

nat,  nit 

i  10 

dit.  Hartley 

where  the  unit  name  Hartley  is  to  honor  the  communication  engineer  R.  V.L, 
Hartley  who  in  192«  introduced  a  logarithmic  measure  of  information  for 
use  in  communication  engineering. 

Let  us  consider  some  of  the  propertiee  of  H  (b). 

(1)  The  vslue  of  H  (b)  is  Independent  of  the  numerical  value,  name, 
quality  category,  ui  other  disignation  of  the  events  Aj,  but  depends  only 
on  the  probabilities  of  their  occurrence. 

(2)  H  (o)  is  a  symmetric  function  of  the  probabilities 

(3)  H  (<(>)  *  0,  with  equality  if  and  only  if  soma  p  *  1  and  all  other 
Pj  *  0,  J  ^  1  (w«  define  0  log  0  =  0). 

(4)  for  two  events,  that  is  c  =  2,  with  p  -  1  / 1  ■  p  ,  H  (b)  *  log  2 
-  1  bit,  that  is 

one  bit  of  information  is  the  capabiiitv  of  resolving  the  uncertainty 
in  a  choice  between  two  equally  likely  alternative • . 

(5)  H  (b)  is  a  maximum  if  all  the  p  are  equal  to  l/c  in  which  case 

H  (b)  =  log  c,  Hartley'*  measure,  1 
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The  maximum  value  of  M  ($)  correspond*  to  the  resolution  of  the 
greataet  uncertainty  and  the  aero  value  of  H  ($>)  corresponds  to  no  un- 
certainty.  From  thle  we  may  infer  an  important  principle,  The 
determination  of  one  of  a  group  of  objects  by  a  sequence  of  selection* 
will  be  most  efficiently  accomplished  in  the  sense  of  minimum  mean 
effort  if  each  selection  is  made  from  equally  probable  groupings,  that 
is,  with  maximum  information,  or  maximum  uncertainty  resolved  for 
each  ealaction. 

Suppose  one  of  k  objects  is  to  be  determined  by  the  answers  to  m 
question*  aa  to  which  group  contains  the  object  in  question,  The  maxi¬ 
mum  information  ia  provided  by  an  answer  if  the  objects  are  grouped 
into  r  groups  of  equal  else  yielding  log,r  bits  of  Information  per 
answer,  The  answers  to  m  Independent  questions  may  be  considered 
as  the  analogue  of  a  communication  channel  of  capacity  C  *  m  log^r 
bits.  The  maximum  uncertainty  in  a  source  of  k  objects  is  H  logjk 
bits.  If  H  s  c,  a  predetermined  object  can  be  uniquely  identified  with¬ 
out  error,  that  is,  the  uncertainty  can  be  completely  resolved,  However, 
if  H>C,  the  identification  cannot  be  made  without  some  possibility  of 
error,  that  is,  the  uncertainty  cannot  be  completely  resolved,  For 
example,  suppose  there  are  27  coins  of  equal  value  of  which  2b  are  of 
the  same  weight  and  one,  a  false  coin,  is  heavier  than  the  others,  How 
many  weighings  on  a  two  pan  balance,  without  weights,  are  necessary 
always  to  determine  the  false  coin?  The  uncertainty  that  must  be  re¬ 
solved  li  H>  log  27,  There  are  3  possible  outcomes  at  a  weighing, 
the  left  pan  is  heavier,  the  right  pan  is  heavier,  both  pane  balsnce. 

Thus  each  weighing  provides  information  of  measure  nt  most  log  3.  To 
be  sure  iu  dele* mine  tie  f»toc  coin  at  least  three  weighings  are  there¬ 
fore  necessary  since  3  lop,  ?  =  log  27.  We  leeve  it  to  you  to  develnp 
11, e  appropriate  sequence  of  operations.  What  if  there  ere  26  coinn? 

Clotcly  related  to  the  preceding  ideas  is  the  coding  theorem  which 
:,tatei  that  given  a  source  of  mean  information  of  H  bits  per  observation 
it  le  possible  to  encode  or  translate  sequences  of  observations  Into 
aequencee  of  two  elements  only,  say  the  binary  digits  0  and  1,  i*m.h  mat 
on  the  average,  a  sequence  of  N  original  observations  (Information 
content  NH  and  information  content  NH  bits.  We  note  this  ae  a  law 
of  conservation  of  information. 
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Suppose  there  ere  k  possible  observations  with  probabilities  of 

)( 

occurrence  pj,  p^,  p^,  X  p^  ■  I.  Each  of  the  k  possible 

observations  could  be  represented  by  a  sequence  of  r  binary  digits, 
where  k  <  2r  or  r  b  loggk,  A  sequence  of  N  observations  would  thus 
be  translated  into  a  sequence  of  rN  e  N  log2k  binary  digits,  The  coding 
theorem  anys  in  effect,  that  by  taking  advantage  of  the  different  probabil¬ 
ities  of  occurrence  of  the  observations  since  -  p  log  p  -  p.iog.p,  •  •  >  • 

L  2  1  2  2  2 

-  logjPj,  “  H*..  lqgj,k,  there  exists  a  more  economical  translation,  NH<  IN 

logjk  *  rN,  and  that  it  is  the  best  possible  translation,  A  similar  notion 
underlies  the  technique  oi  sequential  analysis  in  statistics  which  achieves 
a  test  of  a  certain  strength  with  s  mean  number  of  observations  s<  taller 
than  a  fixed  slsed  sample  of  similar  strength, 

In  particular,  suppose  there  are  four  poestble  observations,  say 
A,  B,  C,  D,  with  respective  probabilities  of  occurrencs  p  (A)  «  1/2, 
p(B)  »  1/4,  p(C)  »  1/8,  p(D)  =  1/8,  For  this  source  it  is  found  that 

H  a  •  l/2.1og  1/2  -  lAlogl/t  -  1/8  log  1/B  -  l/81og  1/8 
(4)  =  l/2log2  +  l/21og  2  +  3/8  log  2  +  3/8  log  2 

■  1,75  log  2  ■  1,75  bits 

A  coding  that-  will  *chl»->e  a  ,,ieun  of  1,75  Unary  digits  pet  original 
observation  is  given  by  the  Fan o  coding 

A  a  0 


D  =  1  1  1 

Note  that  this  coding  is  derived  by  u  sequence  of  groupings  into  two  equally 
probable  groups,  identifying  the  elements  remaining  in  each  group  in  each 
step  by  the  binary  digit  0  or  1;  first  into  the  grouping  (A),  (ft,  C,  D)j  then 
into  the  grouping  (B),  (C,D);  and  finally  into  the  grouping  (C),  (D).  This 
Fano  roding  will  uniquely  convert  a  sequence  of  A,  B,  C,  D's  Into  a 
sequence  oi  binery  digits  and  back  to  the  A,  B,  C,  D's, 


Dcaign  o f  Experiment! 
Ai  an  example, 


(6)  mioioioon  loioooiuono.  .  , 

is  grouped  uniquely  according  to  the  given  Fano  coding  Into 

(7)  1  0/  1  1  0/  1  0/  1  0/  0/  l  l  1/  0/  1  0/  0/  0/  1  l  1/  0/110/, 
and  converted  to 

(8)  BCBBADABAADAC... 

The  mean  number  o £  binary  digit*  per  original  obeervation  ie 

(9)  1x1/2  +  2x1/4  +  .lx  1/8  +  3x1/8  =  1.75 
of  which 

(10)  1  x  1/2  +  1  x  1/4  +  1  x  1/8  «  7/8  ■  0,875 
arc  "0"  ,  and  0,  875  are  "l". 

Far  the  olnary  coding 

A  ■  0  0 

.  B  »  01 

Clio 

Doll 

The  converted  binary  sequence  corresponding  to  N  original 
observation*  would  consist  of  2  N  binary  digit*  rather  than  1,75  N 

binary  digits, 


This  sort  of  idea  is  clearly  of  importance  In  considerations  of 
efficiency  and  economy  not  only  in  communication*  but  the  stores*  vnd 
rapid  scanning  and  retrieval  of  large  volumes  of  data. 

The  facaimila  transmission  of  typewritten  material  by  two  eignal 
levels,  white  or  black,  may  be  accomplished  by  dividing  the  typewritten 
material  into  elements  by  row*  and  columns  100  to  the  inch.  A  particular 
small  sample  of  typewritten  text  yielded  the  following  data  on  the  four 
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possible  pain  of  consecutive  elements) 


Table  2 


Frequency 

Probability 

Binary  Godins 

Fano  Coding 

White  White 

14, 331 
1,786 

0.8209 

0  0 

0 

Black  Black 

0.1020 

1  1 

1  0 

Black  White 

709 

0,0405 

1  0 

1  1  0 

White  Black 

642 

17,  518 

0.0366 

TTWoo 

0  1 

1 1 1 

The  number  of  binary  digits  required  to  record  the  sample  in  binary 
coding,  2  digits  for  each  pair  of  elements  la  2  x  17918  >  35,036)  for  Fano 
coding  the  number  of  binary  digits  required  is 

(12)  1  x  14,381  +  2  x  1,786  +  3  x  (709  4  642)  ■  22,006 

The  number  of  binary  digits  given  by  the  theoretical  H  is  still 
smaller,  16,320,  obtained  from 

(13)  NK  ■  17518  (.0.8209  iog2  0.8209  -  0.1020  log2  0.1020  • 

0.  0405  log2  0.  0405  *  0.  0366  log?  0.  0366) 

■  17518  (0.2335  +  0.3359  +  0.  1B75  +  0,1747) 
a  17518  (o  9316)  -  16,  320. 


For  a  grouping  of  the  elements  into  the  sixteen  sets  of  four 
consecutive  elements,  the  data  gave) 
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Frequency 

Probability 

Binary  Coding 

Coding  -p  loggp 

wwww 

6,  586 

0,752 

0  0  0  0 

0 

0.  3092 

3BBB 

387 

.044 

1111 

1  0  0 

0 

.  1983 

WWBB 

295 

,  034 

0  0  11 

1  0  0 

1 

.1659 

BWWW 

283 

,  032 

10  0  0 

1  0  1 

0 

.1589 

BBWW 

262 

.  032 

110  0 

1  0  1 

l 

.  1589 

WWWB 

264 

,  030 

0  0  0  1 

1  1  0 

0 

,1518 

BBBW 

211 

,  024 

1110 

1  1  0 

1 

.  1291 

WBBB 

171 

.  020 

0  111 

1  1  1 

0  0 

.  1129 

WBBW 

105 

,  012 

0  110 

1  1 1 

0  1 

,0766 

WWBW 

54 

.006 

0  0  10 

1  1 1 

1  00 

.0443 

BBWB 

34 

,004 

110  1 

1 1 1 

1  0  1 

.0319 

BWWB 

31 

,  004 

10  0  1 

1 1 1 

1  1  0  0 

.0319 

WBWW 

31 

.004 

0  10  0 

1 1 1 

110  1 

,0319 

BWBB 

19 

.002 

10  11 

1 1 1 

1110 

.0179 

WBWB 

3 

,  000 

0  10  1 

1  1  1 

1  l  1  1  0  , 0000 

BWBW 

3 

.000 

10  10 

1  1 1 

1111 

1  .0000 

8,  759 

1, 0000 

rrsTVs 

The  number  of  binary  digits  required  to  record  the  sample  in  binary 
coding,  4  dlgita  for  each  of  the  sixteen  possible  groupn  of  elements  is 
4  x  8799  ■  35,038.  For  Fano  coding  the  number  of  binary  digits  required  t* 


(14)  1  x  0586  +  4  x  (367  +  295  +  £81  +  282  +  264  +  211)  +  5x  (171  +  105) 

+  6  x  (54  +  34)  +  7  x  (31  +  31  +  19)  +  8  x  (3  +  3)  -  1  \  997 

The  number  of  digits  given  by  the  theoretical  H  la  8759  x  1,  6195  *  14, 106, 

For  a  grouping  of  the  picture  elements  iuU  the  25b  set*  of  eight  consecu-- 
tire  elements  binary  coding  would  still  require  35036  digite,  but  T'aiio  coding 
require*  only  1  3251  digits,  a  figure  that  more  nearly  approaches  the  theoretic¬ 
al  NH  for  thie  situation,  12869  digits,  Clearly,  as  the  number  of  elements 
grouped  increases,  the  resulte  of  the  Fano  coding  approach  the  theoretical 
value  determined  by  K.  This  is  true  because  grouping  into  equally  likely 
groups  can  be  more  nearly  accomplished  with  the  smaller  probabilities,  To 
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achieve  the  resultant  laving  in  trammieiion,  unfortunately  require!  extra 
complexity  in  the  equipment  for  storage,  scanning,  and  reconversion. 

In  transmitting  sequences  of  binary  digits  whether  in  a  communication 
channel  or  a  computer,  errors  do  occur  and  it  is  therefore  of  interest  to 
be  able  to  detect  and  correct  such  errors  and  moat  desirably  by  the  equip* 
ment  itself.  The  basic  principle  is  to  limit  the  number  of  possible  signals 
that  may  be  transmitted  by  the  use  of  particular  patterns  or  various  kinds 
of  check  or  parity  digits.  The  following  problem  is  mathematically  equiva* 
lent  with  an  error  detecting  and  correcting  code.  Suppose  that  no  more  than 
one  of  seven  electrical  components  say  a  resistor  may  be  defective,  say 
shorted.  How  can  the  possible  defective  one  always  be  detected  in  a 
minimum  number  of  tests?  Hsre  there  are  eight  possibilities,  none  defective 

or  R^,  . Ry  defective.  We  thus  need  log  8  *  3  bits  of  information. 

The  test  can  be  accomplished  by  three  circuits  with  resistors  in  series  and 
testing  each  for  operability. 


rv 

K^, 

k3,  R5 

s  Gj 

(15) 

*1* 

r2, 

R4>  R6 

*  C2 

*«• 

R3' 

V  R? 

1  C  3 

The  defective  resistor,  if  any,  is  deduced  in  accordance  with  the 
foliow'ng  tables 


Nnnope  ruble 


C 


i 


c 

c 


2 

3 


Defect:.  »r 


and  Cg 
C  ^  and  C  3 
C2  and  C ^ 

C1  end  C2  and  Cj 


ft.. 


R 

R 

R 


3 

4 
1 


Table  4 
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Of  course  if  *11  the  circuit*  are  operable  then  there  i*  no  defective  resistor. 


Let  u*  return  to  the  c -category  ayitem  9  .  Clearly  H  •>  •  Lp^log  pj  i* 
a  parameter  of  thi*  ayitem  of  probabilities  or  of  the  corresponding  multi¬ 
nomial  distribution  were  we  to  take  a  random  sample  of  n  observations  from 
♦Vie  c-category  population,  A  measure  of  the  mean  divergence  per  observa¬ 
tion  of  the  distribution  (pj,  p  ;  , . . ,  p^)  from  the  uniform  distribution 

(l/c,  1/c,  , . . ,  1/e)  or  the  redundancy  of  the  communication  engineer  in  the 
■Ignals  Aj . Ac  is 

c  c 

(16)  log  c  -  H  ■  log  c  +  ,S  p  log  p  *  t  p  log  p./(l/c), 

i»l  1  1  M  »  1 

and  for  n  independent  obeervatione  it  la 

(17)  1„  «  Pjlog  PjAl/c), 

We  may  verify  thie  as  follow*,  There  are  cn  possible  samples,  The 
entropy  for  the  sample  distribution  from  the  population  (p^  p^,  , ,  , ,  pp) 

is 


H«  "  *  C  pit  •pinluB  Bil  '  ’  ‘  P'n 


■  -  £ _ ill 


nc  ni 
pc  l°B  1 


“ (p^og  Pt  +  ...+pclogpc)  ==  „H 


where  in  the  first  summation  (ij  ,  . , , ,  i^)  varies  over  all  possible 
permutations  of  (1,  2,  . , , ,  c)  taken  n  at  a  lime  and  n  +  n  +  .  ,  , 

+  nc  ■  n  with  the  number  of  oceurwncse  of  category  1  in  a  sample, 

The  entropy  for  the  sample  distribution  for  the  uniform  population  is 
log  cn  «  n  log  r 
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Given  e  sample  of  n  observations,  with  ,  n 

1  c 

respectively  In  each  of  the  c  categories,  n  °  £ 
p's  and  XR  in  (17)  above  by 


2‘ 


, .  ,,n  occurrences 
c 

we  may  estimate  the 


(19)  Pj  •  ttj/a,  »»  log  PjAl/c). 


It  turns  out  that  2  l'n,  using  natural  logarithms,  asymptotically  has  a 

chi-  square  distribution  with  c-1  degrees  of  freedom  under  the  null  hypothesis 
of  a  uniform  distribution,  If  the  sample  were  drawn  from  the  population 

(P}>  •  •  • ,  p  )  then  2  1R  asymptotically  has  a  noncentral  chi* square  distribution 
with  c-1  degress  of  freedom  and  noncentrality  parameter  2  X  , 

Let  us  now  consider  the  case  of  the  system  4  in  which  aach  evsnt 
corresponds  to  the  Joint  occurrence  of  a  pair  of  values  say  a  (o^,  0j) 

1  ■  1 . .  j  =  1 . .  with  the  corresponding  joint  probabilities  J 

P  ( « i,  fij),  with 

m  sa  p(«lt  Pp  .  i. 

■  J 


There  now  enter  into  consideration  the  marginal  probabilities 


(2D  r(Tl)  °  f  pK.  3,)  ,  p(U()  .  f  p(0i(  0^) 

and  the  conditional  probabilities 

(22)  p  (o^  |  3^)  «  p  <«J.  3j)/ P  (3j)  :  p  (3j  I  <*j)  *  p  (bj,  Py/  y>  (ofj), 

V/c  define  the  entropy  for  each  of  thu  set*  of  joint,  margliip.l  and 
conditional  probabilities  by 
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H  (a,  3)  ■  -  ^  j:  p  (oij,  3j)  log  p  (ot^  3j>  , 

H  <«)  -  -  jp  p  (ffj)  log  p  (oij)  ;  H  (0)  ■  ■  I  p  Wj)  log  p  Oj ) 

*23*  K/rtl^j  -  -  r  pittj  |  frj)  log  p  |  ^ ) 

H<ff|3)  -  t  P<3J  H  (t»  |  p  ) 
i  1  3 

H  (3  I  «i>  *  -  £  p  (3j  I  «t)  log  p  (3j  I  «j) 

11  (3  I  « )  ■  f  p  (oij)  H  (3  I  a4) 

It  may  be  ehown  that 

(24)  H  <«,  3)  ■  H<«>  +  H  (3  |  «> 

-  H(3)  +  H  <<r  |  3> 

that  i*,  the  mean  uncertainty  about  (he  pair  (a,fi  )  ia  the  mean  uncertainty 
*bout  a  plus  the  mean  uncertainty  about  3  given  o  ,  with  a  eimilar  interpret* 
Uon  for  the  eecond  equation.  It  may  alao  be  ehown  that 

U:)  H(3  I  a)  <  H<3)  i  H(«  |  3)  *  H  («) 


with  equality  if  and  only  if  »  and  P  are  independent,  that  1*  p  (  <*ji  3j  )  " 

p(ctj)  p  (3j)  •  Note  that  (25)  mean*  that  the  mean  uncertainty  about  3 
a  ie  lee*  than  the  mean  uncertainty  about  3  and  the  two  arc  nqutl  If  anti 
only  if  a  ia  independent  of  3  and  contribute*  no  information. 

from  (24)  and  (25)  we  aec  that 


(26)  H  (a,  3)  *  H  (or  >  +  H  (3) 


with  equality  if  and  only  if  o  unci  3  are  independent. 
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If  a  is  the  input  to  a  communication  channel  and  0  i»  the  output,  then 

(27)  H  to)  -  H  («  |  3)  «  I  to.  0)  *  0 


i*  a  measure  of  the  information  transmitted  via  the  channel  or  of  the 
statistical  dependence  between  output  and  input,  It  may  be  seen  that 


(28)  I  to,  0)  «  H  to)  +  H  (0)  -  H  to,  0) 

«  Ptot)  log  ptot)  -  E  P<0j)  log  p (0^) 

+  ft  P  tot,  0  >  log  p  tot,  0  ) 

■  E  £  P  toi,  0j)  logf-P^i;  V  )  , 

1  J  \  p  to  ^  p  (0j)' 

Mathematically,  the  channel  is  characterised  by  the  conditional 
probabilities  p  (0j  j  a^)  .  We  note  that 

(29)  lto,0)  •  £  p  tot)  E,  p  (0^  |  at>  log  -  pg(g'T~ 


and  tl.o  capacity  of  the  channel  is  defined  as 


(30)  C  ■  sup  I  to,0) 

where  the  sup  Is  over  all  possible  inputs  {pto.)j  givan  jp  (0.  |  ftj)j,  and 
we  recall  that  p  (0^)  ■  |  p  <0j  I  «j)  p  to^). 

Thu  expression  for  1  to,0)  a*  written  in  (2H)  is  u  special  case  of 
the  more,  general  result 

(31)  KXjY)  -  /  r(x,y)  log  j-fxt’hCy)  d  A  W  M  (y) 

which  is  itself  a  special  case  of 

tl  (?) 

(12)  1(1:2)  *  /  r,  (?  )  log  "j — jjiy  d  \  (?) 

2 
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where  l (?)  end  f  (?)  are  the  generalised  densities  corresponding 
to  th»  distribution  of  the  random  variable  ?  under  the  hypotheses 
and  Hj  •  In  the  expression  for  I  (X,  Y),  H,  is  the  hypothesis  that  the 
component  random  variables  in  ?  a  (X,Y)  are  dependent  and  H  is  the 
hypothesis  of  independence.  We  note  that  ?  may  be  a  random  vector  of 
n  components  or  even  a  stochastic  process  and  the  lima  remark  holds  for 
X  and  Y  in  (31). 

In  Ihe  remainder  of  the  dlieussion  we  limit  ourselves  to  natural 
logarithms.  We  mention  two  interesting  particular  cases  of  (31). 

(1)  if  X  and  Y  arc  normally  distributed,  then  it  is  found  that 
(33)  1(X,Y)  »  -  1/2  log(l  -  p2) 

where  p  Is  the  correlation  coefficient  of  X  and  Y  and  we  note  that  (33) 
does  not  depend  on  tho  means  or  varlancee, 


(2)  Let  Y  be  a  parameter  0  ranging  over  a  apace  ®  >  so  that 
!  (x,  0)  ie  the  joint  probability  density  of  x  and  0 ,  h  ( (»  )  is.  the  prior 
probability  density  of  9  ,  g  (x  (  6)  is  the  conditional  probability  density 
of  v  given  0,  and  the  marginal  probability  density  of  x  is  g  (x) 

* J  g  (  x  |  0  )  h  (  0)  d  0.  An  experiment  is  the  ordered  triple  (Xi®  > 
g  T*  |  N  )1  a  (  and  tbr  information  provided  by  the  experiment  |  , 
with  prior  knowledge  U  (  0)  is 


(3.i)  us)  a  //  i(x,0)  log  -|xj - }  ax  da. 


We  shall  not  consider  the  many  Interesting  end  oaaful  properties  of 
1  (1:2)  In  (32). 

We  shall  now  give  a  simple  Illustration  of  the  additive  analysis  of 
the  information  measures  and  its  applicstion  in  tests  of  hypotheses.  It 
may  be  easily  shown  that 
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(35)  l(X,Y,Z)  -  I(X,(Y,Z)  )  +  I(Y,Z) 


where 

(36)  I  (X,  Y  i  Z)  =  j’iU.y,*)  log  —  d  X(x)d^(y)  d^(») 

end 

(37)  1  (X, ( Y,  Z)  )  ■  j*  f  (x,  y,  *)  log  f(x)f(y,e)  d  ^  «i^(y)  d\>(») 

that  i ■ ,  the  meaeur*  oi  th*  mutual  independence  of  X,  Y,  end  Z  le  enelyeed 
Into  a  meaaure  of  th*  Independence  of  X  end  the  pair  (Y,  Z)  and  th*  meaeure 
of  the  Independence  of  Y  and  Z, 

For  eample*  of  eie*  n  from  normal  population*  th*  aetlmato*  of  the 
vu.li! i, «  in  (35)  lead  to  the  analyeia  oi  information  table  5 


Iniorrnation  Component 


-  (n-1)  log  (1  "rym  ) 

1 

3/2M 

-  (n-1)  log  (1-r*  yB) 

2 

8/2M 
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where  the  degrees  of  freedom  ere  those  of  the  noncentral  chi-squere 
distribution  with  noneentrallty  parameter  r  *  under  the  null  hypothesis  of 
independence. 

For  a  three-way  rxcxd  contingency  table  the  corresponding 
analysis  becomes  that  in  table  6, 


Table  6 


Information  Component 


2f  Ujlt10* 


•JL 


f-J.  f..k 


n  f 


2  S  a  £  f  log  — 
i  j  k  ijk  *  £. 


iiiL 


D,  F, 
(cl)  (d-1) 

(r-1) (od-1) 


?  ??*, 


ijk 


log 


n2f 


■t,  f 


il)L 


J. 


rcd-r-o-d+2 


where  f.  .  is  the  frequency  of  occurrence  in  the  cell  at  the  i-th  row, 

j-rn  column,  and  k-th  depth  with  f  ■  £  f  ,,,  ,  f  ■  ?  £  f.  , 

•  J  k  1  ij  k  i ,  ,  j  «  1  j  k 

n  -  1)  S  1  f...  etc.,  and  the  degree*  of  freedom  are  those  of  the 
1  j  k  ijk 

asymptoti :  chi-square  distribution  under  the  null  hypothesis  of  independence 


We  remark  that  there  are  many  othor  applications  cf  these  lout 
concepts  but  we  lln.it  ourselves  to  thsse  two  becauss  of  time  Also  that 
more  extensive  tables  of  the  noncentral  chl-»quare  distribution  have 
recently  been  computed  than  those  first  publishtd  by  R.  A.  Flshs'.  U 
may  bs  noted  that  all  the  expressions  in  the  contingency  table  analysis 
may  be  expanded  Into  sums  and  difierences  of  terms  of  the  form  2nlog  n. 
tables  of  2nlog  n  are  available  for  n  *  1  (1)  10,  000  so  that  the  arithmetic 
is  one  of  table  look-up  addition  and  subtraction. 

I  K>-' 'i. fit  this  exposition  has  stimulated  some  of  you  to  look  further 
into  throe  matter*  as  possibly  useful  in  your  tir-vie  of  application. 


THE  CONCEPT  OP  MONOTONE  FAILURE  RATE 
IN  RELIABILITY  THEORY 


Frank  Pro* chan 

Boeing  Scientific  Research  Laboratoriei 


1.  INTRODUCTION,  Moat  analyaea  of  reliability  problem!  assume 
that  the  form  of  the  underlying  failure  distribution) ■)  ia  known)  the  para* 
mater  may  be  aaaumed  either  known  or  unknown.  Popular  families  of 
failure  distributions  are  the  exponential)  Weibull,  gamma,  normal,  and 
lognormal, 

The  weakness  of  such  analyses  is  that  the  conclusions  reached  may 
'b'ls  grossly  •’in  error  if  the  assumption  as  to  underlying  failure  distributions 
is  incorrect.  In  fact,  the  error  in  the  original  assumption  may  be  greatly 
compounded  in  arriving  at  the  final  conclusion,  especially  if  the  eonolualon 
concerns  a  tail  probability  or  if  the  System  analysed  is  complex,  A  good 
example  of  the  pitfalls  of  an  erroneous  assumption  as  to  the  form  of  the 
underlying  failure  distribution  is  furnished  by  Zslen  and  DannemUler  (1961). 
They  Investigated  the  robustness  of  four  representative  acceptance  sampling 
procedures  derived  from  the  exponentiel  distribution  when  in  fact  the 
failure  times  followed  a  Weibull  distribution  with  the  same  mean  life. 

This  was  done  by  constructing  the  operating  characteristic  (O.  C. )  curves 
for  these  procedures  wnen  the  parent  distribution  of  failure  times- was 
the  Weibull  distribution  with  survival  probability  expl-t P/©)  for  values 
of  the  shape  parameter  p  ■  3/2,  2,  end  3.  To  illustrate  how  thoroughly 
er.  oncous  a  conclusion  can  be  when  baaed  on  an  incorrect  assumption 
as  to  the  form  of  the  underlying  failure  distribution,  we  .^produce  the 
O.  G.  curves  for  s  censored  non-replacement  plan  (n  ■  28,  r  a  14)  bated 
nn  the  exponential  distribution  when  the  parent  distribution  1*  actually 
Weibull  with  survival  probability  exp(-tP/B),  A  censored  non-r«piac.T'nnt 
phr,  consists  of  placing  n  items  on  test  simultaneously  and  stopping  the 
test  after  the  failure. 

Note  that  while  the  probebility  of  acceptance  aeeumlng  an  exponential 
distribution  with  mean  S00  is  only  .  10,  the  corresponding  probaDillty 
assuming  a  Weibull  distribution  with  the  same  mean  is  assentiaily  1. 

Thus  if  a  mean  life  of  300  hours  were  unacceptable,  "bed'1  lots  would 
always  be  accepted  under  the  sampling  plan  il  the  distribution  were 
Weibull  with  shape  parameter  3,  instead  of  being  rejected  90%  of  th<>  time 
ae  they  should  be  if  the  dietribution  were  exponential  with  the  same  mean. 
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A  further  embarrassment  may  face  the  reliability  analyst  who  attempts 
to  make  specific  assumptions  as  to  the  form  of  the  underlying  failure 
distributions.  Ho  just  may  not  have  (sufficient  relevant  information,  since 
in  general  many  observations  are  required  lo  make  reasonable  Inferences 
about  the  form  of  a  distribution,  as  compsred  to  the  number  required  to 
eutimate  a  parameter,  such  as  the  mean.  As  an  example,  we  summarise 
the  results  of  an  investigation  undertaken  by  Zeien  and  Dannemillor  (1961) 
to  see  how  well  two  statistical  tests  described  by  Epstein  (I960)  would 
distinguish  betweun  the  exponential  and  the  Welbull  distributions.  Consider 
a  non-replaccment  censored  ulioatlon  where  the  hie  test  starts  with  n 
items  and  ends  as  soon  as  the  first  r  failures  occur,  Assuming  failures 
occur  at  times  T2«;  .  .  ,  <Tr,  the  total  time  on  test  up  to  the  i*" 
failure  is 

T(Tt)  »  Tj  +  .  .  ,  +  tt  f  (n  .  I  +■  DTj,  i  •  1 . . 

Then  the  conditional  distribution  ofTlT^),  T'lT^)  ,  ,  ,  .  ,  T(T^  ^)  is  uni* 
form  over  [O,  T(Tp)j  for  fixed  T(Tr)  .  Hence 

r-t 


V'’T(Tt)  -i:l_.T(Tr) 


Is  an  approximate  normn  deviate,  Epstein's  Test  3  calls  for  rsjectlon 
at  levels,  if  |z]  >  where  is  the  100(1-  <X/l)  perc#,,inne 

point  ul  the  standard  normal  distribution. 

Epstein's  Test  8  uses  tne  Earlett  statistic 


X  n  -  “  [in  T(TjH  (t(T2)  ••  T(T ,)}+...  +  t)-T(TM  jj 


W  =  2r 


i  +  .Ltl 


tr 


i 
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l 

asymptotically  distributed  as  s  %  variable  with  r  ■  1  dagraas  of  free- 
tiom.  Tha  tast  proeadura  Is  to  rajact  at  laval  cL  If  W>  %J[(r  •  1). 

To  dstsrmlne  tha  proportion  of  timas  tha  tasts  would  distinguish  ba> 
twatn  tha  exponential  and  Wetbull  distributions)  Zalan  and  Dannsmillar 
ran  an  ampirlcai  sampling  study  on  a  computer.  Samples  wars  drawn 
from  each  of  tha  Wcibull  distributions  with  shape  parameter  p  •  3/2  ,  2. 
and  3  and  from  tha  axponantial  distribution,  all  having  mean  one.  Tha  two 
tests  ware  applied  at  level  of  significance  >03,  The  results  werst 


f  Proportion  of  timas  axponantial  assumption  rejected 
at  .  05  laval  of  significance 


Test 

P  ... 

n  a  14,  r  a  14 

n  a  25,  r  a  14 

n  ■  25,  *..■  ji 

3 

1 

.057 

.050 

3/2 

.  340 

,  253 

2 

.  837 

.643 

3 

1,000 

.983 

8 

1 

,  043* 

.063 

Mggm 

3/2 

.063* 

,  080 

2 

.  178* 

.  117 

3 

.  575* 

.  420 

.  S4if* 

Stirred  values  are  based  on  600  samples  or  simulation...  All  other 
values  are  based  on  300  samples. 

t  Reproduced  from  M.  Zalan  and  M.  C,  Dannsmillar,  "Tha  robustness 
of  life  tasting  procedures  derived  from  the  exponential  distribution," 
Tcehnometrlcs,  Vol.  3,  No.  1,  February  1961, 


The  conclusion  seems  clear  that  for  small  sample  sieee  it  Is  difficult 
to  distinguish  between  the  Weibull  and  the  exponential. 
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What  l«  nestled  th*n  in  reliability  art  analyses  baaed  not  on  specific 
familial  of  underlying  failure  distribution!,  but  rather  on  broad  assump- 
tione  concerning  failure  corresponding  to  the  aetual  physical  situation.  One 
very  natural  assumption  of  this  type  is  that  corresponding  to  a  failure  dis¬ 
tribution  F  with  density  f  and  eurvival  probability  F  ■  1-F,  the  condition¬ 
al  failure  r  ate 


(1.1) 


r(t)  «  iiiL 

F(t> 


defined  for  F(t)  >  0  la  increasing  (decreasing)  with  time  t.  0<t<  oo  , 
Physically.  r(t)dt  is  the  conditional  probability  of  failure  in  t>  t+dt, 
given  survival  until  time  t.  We  may  verify  by  differentiation,  that 
under  the  assumption  of  increasing  (decreasing)  failure  rate  the  quantity 
l  n  F(t)  is  concave  (convex)  on  [0,  oo).  More  generally.  Whether  a 
density  exists  nr  not.  wejhail  say  a  distribution  has  an  increasing 
failure  rate  (IFR)  if  fa|(t)  is  concave  on  £o,  ),  and  a  decreasing 

failure  rate  (DFR)  if  J(n  F(t)  is  convex  on  [0,  oo). 


Examples  of  IFR  and  DFR  distribution ■  aret 
(1)  The  exponential) 

f(t)  ■  Xexp(-Xt).  t  >  0. 

Both  IFR  and  DFR. 


(it)  The  gammas 

f(t)  ■  X(Xt J**1  expt-Xt^),  C*>0,  t>0. 
IFR  for  <x  >  1,  DFR  for  0(<  1. 

(ill)  The  Weibullt 

f(t)  ■  “l  exp( -  Xt*  ),  COO.  t  >0. 

IFR  for  DFR  for  O.  <  1. 

(iv)  The  truncate*  normal! 

1 


«t>  • 


a  cr  VTrT 


exp 


/.  ILliiil] 

1  2  cr*  J 


O'>0,  0  <  t<«,-‘, 


where  a  Is  a  normalising  constant. 
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Physically,  IFR  might  correspond  to  wsarout,  so  that  the  older  the 
item  gate,  the  greeter  ita  chance  of  failure.  Example*  are  rubber  tire*, 
human  being*  pait  some  initial  period,  and  many  mechanical  part*  which 
gradually  wear  out.  Physically,  DFR  might  correspond  to  work  hardening, 
ho  that  the  older  the  Item  getu  the  tougher  it  gets,  and  hence  the  lea*  chance 
it  ha*  of  failing.  Certain  metal*  act  in  thl*  fashion.  In  Proachen  (196J) 
i*  discussed  another  commonly  occurring  process  producing  DFR 
distributions. 

How  far  can  we  get  in  reliability  analyses  assuming  IFR  (DFR)t  In 
the  following  sections  we  hope  to  show  that  a  rather  surprising  number  of 
useful  results  and  methods  follow  from  this  modest  and  natural  assumption. 

2,  BOUNDS  ON  QUANTITIES  OF  INTEREST  IN  RELIABILITY  THEORY 
ASSUMING  IFR  (DFR),  A  number  of  interesting  and  Informative  bounds 
can  be  obtained  on  quantltlss  arising  frequently  in  reliability  problems 
If  we  assume  IFR  (DFR)  distributions.  In  this  section  wo  survey  some  of 
the  more  simple  and  useful  bounds.  The  reader  is  rafarrad  to  Barlow 
and  Marshall  (1963a.  and  b),  Barlow  and  Proschan  (1963),  and  Barlow 
and  Proschan  (1964),  for  details  and  further  bounds, 

Since  the  exponential  distribution  with  constant  failure  rata  is  boundary 
distribution  between  IFR  and  DFR  distributions,  it  provides  natural 
bounds  on  the  survival  probability  of  IFR  and  DFR  distributions. 

Thsorsm  1. 1,  JJ,  F  ,1*  IFR  (DFR)  and  F(  j  )  ,  p,  i,  a.  .  ,r  it  ± 
ntn  ps r cantlls,  then  ** 


(2.  i) 


» 
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wVuj  re 

CL  .  . 

«.P 

Proof,  Sine#  -in  F(t)  i*  coneive,  (convex)  »nd  F(t)  is  decreasing) 

then  An  F(t)  •  4n  F(O’)  it  decreasing  (increasing).  Consequently 
t  -  0 

^F(t)}1^4  it  dec  ratting  (Inerts  sing),  Thus  F(t)'^>  (i  *  p)‘^P  for 

t<  g  ,  snd  F(t)1/l<  (1  -p)WiP  for  t>  \.,\\ 

P  (£) 

Thus  if  we  know  s  percentile  of  tn  IFR  (DFR)  dittributlon  ws  Immedi¬ 
ately  have  one- tided  bounds  on  the  survival  probability  corresponding  to 
each  instant  of  time,  What  if  wt  know  the  mean  of  an  IFR  distribution? 
Then  Theorem  2.  2  provides  a  lower  bound  on  survival  probability. 

Theorem  2.  2.  I£  F  is  IFR  with  npoan  then 

(2.  2) 

F(t)> 


The  'naquality  it  sharp, 

See  Barlow  and  Marshall  (1963a)  or  Barlow  and  Frosnhan  (1964). 
Chaptor  II,  for  the  proof.  Note  that  the  exponential  distribution  with  met’, 
attaint  the  lower  bound  for  t<  while  the  degenerate  distribution 

concentrating  at  attaint  the  lower  bound  for  t>|ij. 


{  **^1 


<Ki 
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An  obvioui  application  of  Theorem  2.  2  li  to  a  eyetem  consisting  of  n 
Independent  component*  In  eerie*  with  distributions  Fj(IFR)  with  moan* 

^ ljU  ■  1,  2,  .  .  .  ,  n).  Then  using  (2.2)  the  systsm  survival  probability  1* 


TTn...  j 


I  £  1  /M.i  t < mini 


elsewhere. 


Ths  bound  is  sharp.  This  indicate*  why  system  reliability  is  often  better 
than  predicted  on  a  part*  count  basis.  Likewise  for  a  parallel  system 
with  component  distributions  Ft  with  means  |A  (i  ■  1,  2,  .  .  .  ,  n), 


lW>  « 

1*1 


(i  -  • 


t<mln  (  jUj,  ....  M-r> 


elsewhere. 


In  fact,  the  application  of  Theorem  2.  2  can  be  extended  to  so-called  mono* 
t’.uic  structures.  (See  Barlow  and  Froschan  (19b4),  Chapter  VH,  Theorem 
3  1.)  To  define  *  mr.n3tr.nie  structure  we  let  xj  *«piae*nt  the  state  of 
the  1th  component  of  the  stn'^ture,  with  x,  ■  1  if  the;  component  is 
functioning  and  x^  ■  0  if  the  l*“  component  is  not  functioning,  i  »  ! ,  2,  . , 
...  n.  Let  4(*i'  x2>  *  1  1  •  *n)  represent  the  corresponding  state  of  the 
structure,  with  (I  n  1  if  the  structure  is  functioning  and  i  ■  0  It  the 
structure  Is  not  functioning.  The  structure  is  said  to  l>*  mono  tonic  if 
$(Xj,  Xg . xn)  is  monotonie  increasing  In  each  argument  with 

0(0,  0 . 0)  ■  0  and  $(l,  1 . 1)  ■  1.  Intuitively,  s  monotonic 

structure  is  on*  diet  performs  st  least  as  wall  if  felled  components  srs 
replaced  by  functioning  components.  Most  reasonable  structures  occurring 
in  practice  will,  of  course,  b«  monotonie.  8e*  Blrnbsum,  Esary,  and 
Saundars  (1961)  for  a  discussion  of  monotonie  structure*  (called  by  them 
"coherent"), 
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Next  suppoia  etochaeucally  independent  component*  with  the  relie* 
bility  of  the  ith  component  p^,  i  ■  1 ,  2,  ,  .  ,  ,  n.  Then  the  reliebility  of 

the  ey item  will  be  rapreientcd  by  h(pl(  p2 . pj,  In  the  epeclel 

case  Pj  ■  Pj  ■  .  •  ■  ■  pn  ■  p,  we  will  write  for  convenience  h(p), 

Barlow  end  Fro ichtn  (1064))  Chapter  VII,  ehow  that  for  monotonic 
etructure*  h(pi,  p-, . pn)  it  monotonic  in  each  argument, 

Ueing  thie  fact  we  may  obtain  the  following  lower  bound  on  ayetem 
reliability  (or  monntonic  etructure*. 


Theorem  2,  3,  Let  F^,  IFR  with  mean  |X^(  i  *  1,  2. 


tii  be  the 

HUM  II  ■■■! 

Then 

t)) 


n' 


failure  dietributlone  of  the  component*  of  ^  monotonic  etructure 
for  t<min(  jWj  ,  .  .  ,  .  jx  ),  the  eyetem  reliability  h('fl^tt) ,  TV 
-t/jtu 

>h(e  . .  n),  the  correeppndlOB  eyetem  reliability  when 

the  component*  arc  exponentially  dletributed  witn  correepondlng  mean*. 


Proof,  Since  n(pj,  .  .  , ,  pn)  i»  monotonia  lncreaeing  in  each  argu¬ 
ment  and  by  Theorem  2,  2  for  t<mir(|U.j,,  ,  /Xft)  i  F^(t)  See”4'^, 

1*1 . .  then  it  follow i  that  for  t  <  min(  /U.^,  ,  ,  .  ,  fi^), 

-  -  "t/M-i  -t/u.  h 

h(Fi(t) . Fn(t))>h(e  . .  n),|| 

We  may  apply  Theorem  2.  2  to  the  n-fold  convolution  F  *^(t)(  where 
7  le  IFR  with  mear,  to  nbtaln 

i  \  n„i  (t/ 

F  n  ft)  <  i  -  Z  - r. - exp<  •  t/  jU. .) 

|«0  ’ 


for  t <  Therefore,  if  N(t)  l*  the  number  of  renewal*  in  [o,  tj  of  a 
renewal  proceee  bared  on  F, 

<  L  — pf”1—  **!»(  -  t /  /Lj) 

(an 


* 
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for  t  <  |ui  .  Thu*  w*  have  tha  elementary  but  important  reeuit  that  under 
tha  IFR  assumption  tha  Poiiaon  di  atrlbution  provide  I  a  oonoarvatlv*  eetl- 
mat*  of  tha  probability  of  n  or  mora  failure*  in  f 0 ,  t]  for  t  lea*  than  the 
mean  life  of  a  aingle  component.  ■  J 

Thii  reault  haa  application  in  the  following  apara  part*  altuation. 
Aaaume  that  wa  have  on*  type  of  tub*  in  n  sockets  which  ia  replaced 
immediately  upon  failure.  Let  Nj(tj)  denote  the  number  of  failures 
occurring  in  the  socket  before  time  tj,  the  time  tha  }tl1  socket 
ia  to  remain  in  operation,  if  the  sockets  era  'stochastically  independent 
and  the  life  distribution!  are  exponential  with  parameter  X. ,  then 
Nj(tj)  Nn(tn)  la  a  Poisson  random  variable  with  parameter 

<1  »  \  5"  t.i  and 

J-l  j 

p[N1(t1)  +  .  .  .  ♦  Nn(tn)<N]  ■  ^1.  ' 

Cramer  (194>),  page  £05,  If  all  we  can  aaaume  is  that  F  la  irR  with 
mean  1/^  ,  and  each  tj  <1  /\  ,  then 

*(n,<V  +  '  .  •  ♦Nn(tn)^N]>  ^i-e“#. 


Using  this  bound,  N.  the  number  of  sparse  to  be  stocked,  cap  Ha  chosen 
so  that  wi  will  be  protected  with  high  probability  again*!  a  shortage  e»' 
spare*. 

The  beat  upper  bound  on  F(t)  when  F  is  IFR  I e  given  by  the 
following  theorem. 


Dtiign  ol  ExperlminJ*  ; 

’n*A« 


Theorem  Z.  4.  &  V  i.  XFR  withm«!i  Ml  SiSL 


?U)< 


aru-i-r 
:  ■  TcrJv*3 


-  Wt 

whtre  co.  deptndlng  on  t,  ifctif.flu.  1  •  •  •  •  .fS 

PtfOP.f*  ki  ^  i 

*<*  Sil' 

QU)  -  ...  IfiS 

0  *-t  ipfc.  . 

Th.n  inO(x)  (linear  on  fO.t))  «»•»•  ±nT(*)  (eoneav.  pn  [0, 

Lh”h,  0<»<t  Thu,li(.)  nan  rw  MS. 

0<x<t,  See  rtgure  2,  1.  t  >•  ■  Igy 

U  »  m  can  nl»nyi  *****  *  ■»  *•»  •' 

Foe  thin  cholcn  o.  U.T«  n......rily. ««••  •■“*  •»»«»  »**". 

h-onn  *hove  tor  0 x<t.  Therefore  T(t)<  •  unli««  T  owi*iPP| 
ld.Mi.Mlly  with  O.  Sine,  a  11  iru,  the  bound  1*  *h*rp. 

For  t<  a,  th.  In. quality  1.  ohvtou*.  Th.  degenerate  dl.trityiiWlI 
concentratinj  at  Mi  proviU..  th.  upper  bound  tor  t<  p.,'  || 

Not.  that  a.  t-yoe,  »-*!/  Mr  “  «n  b#  ,bown  th,U  th*  Upp,r  ^ 
In  Theorem  2,4  i«  aeymptotie  to  .-t  H*1. 

Figure  2,  2  lllu.trata.  th.  bait  upp.t  and  lower  bound#  on 
wh.rv  T  1.  IFR  and  fa  -  1.  Table  I  tabulate*  th.  upper  bound,.  ■ 

We  can  alio  obtain  bound*  on  percentile*  In  t.rm*  of  th.  mean 
vie.  veriai 
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TABLE  I 


UPPER  BOUNDS  ON  1  -  F(t) 


CO 


(  F 

Is  IFR, 

u  =  5  tdF(t) 

X  A 

=  1) 

IFR 

Markov 

0 

IFR 

Markov 

t 

Bound 

Bound 

t 

Bound 

Bound 

(1/t) 

(l/t) 

1.0 

1.000 

1.000 

1.1 

0.820 

0.909 

3.1 

0.053 

0.323 

1.2 

0.637 

0.833 

3.2 

0.047 

0.313 

1.3 

0-577 

0.769 

3.3 

0.042 

0.303 

1.4 

1.5 

0.488 

0.416 

0.714 

0.667 

3.4 

3.5 

0.038 

AO  i 

u*  w** 

0.294 

0.286 

1.6 

0.358 

0.625 

3.6 

0.030 

0.278 

1.7 

0.308 

0.588 

3.7 

0.027 

0.270 

1.8 

0.268 

0.555 

3.8 

0.024 

0.263 

1.9 

0.233 

0.526 

3-9 

0.022 

G.  256 

2.0 

0.203 

0.500 

t  r\ 

*+•  V 

0.020 

0.250 

2.1 

0.178 

0.476 

4.1 

0.013 

O.  2.44 

2.2 

0.156 

0.455 

4.2 

0.C16 

C.236 

2.3 

0.138 

0;435 

4.3 

0.014 

0.233 

2.4 

0.121 

C.417 

4*4 

0.013 

0.22/ 

2.5 

0.107 

0.400 

4.5 

0.012 

0.222 

<■6 

0.095 

0.385 

4.6 

n  nm 

V  •  v/j.  A 

0-23.7 

2.7 

0.084 

0.370 

4.7 

0.010 

0.213 

2.8 

0.075 

0."^7 

4.8 

C.009 

0.2CS 

2.9 

0.067 

0.347 

4.9 

0.008 

0.204 

3.0 

0.059 

0.  ;  " 

5.0 

0.007 

0.200 
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Theorem  2.  5.  A asume  F  is  IFR.  For  p<.  1  -  e 

[-  P)  ]  P  <  5P<  [  -  *"p'  •pl  ]  P-p 

for  p  "J>  1  -  e  1 


where 

H,p  =  sup  (t  :  F(t)  <pj  . 

The  inequalities  are  sharp. 

See  Barlow  and  Marshall  (1963a)  or  Barlow  and  Proschan  (1964), 
Chapter  II,  for  a  proof. 

In  life  test  sample?;  some  items  may  not  fail  at  all  during  the  course 
of  the  test.  Therefore  the  usual  sample  average  cannot  be  used.  However 
some  percentile  estimates  will  always  be  available.  Note  th  using 
Theorem  2.  5  we  can  obtain  bounds  on  the  mean  of  IFR  distributions  in 
t»rms  of  percentiles.  For  example  if  M  is  the  median,  then 

M  M 

2/.n2 


For  r  DFR  upper  bounds  on  F  (t)  .%in  be  given  in  tenriB  of  the  mean 
as  shown  in: 


Theorem  2.  6.  If  F  is  DFR  with  mean  then 


t< 


^  ft 


F(t)  <  J 


ft 


t  > 


^  Ml 


Preceding  Page  Blank 
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The  inequality  is  sharp. 

See  Bar'.jw  and  Marshall  1 963a)  or  Barlow  and  Pro  schan(  1964) ,  Chapterll, 
for  a  proof. 

The  following  lemma  will  pro-  ice  us  with  useful  moment  inequalities  and 
further  comparisons  with  the  exponent^ 


Lemma  2.  7.  If 

(a)  F  is  1FR  with  mean  m  ....  .  .  ,(x) 

(b)  (j)  (x)  is  increasing  (decreasing), 
then 


-  e 


/  4>(x)F(x)dx_< 
(2l> 


(x)G(x'dx. 


Proof.  Suppose  <p  is  increasing  and  F  is  not  identically  equal  to  G. 
Since  F  is  IFR  and  G  is  the  exponential  distribution  with  the  same  mean, 
F  crosses  G  exactly  once  from  above  at,  say,  t^;  F(t^),  =  G^q).  Then 


fQ4>(x) F(x)dx  -jfc£>(x)G(x)dx  =  j[j$(x)  -  <f)  (tQ)] 

[_F(x)  -G(x)jdx<.  0. 

lo  obtain  the  conclusion  for  <£)  decreasing,  replace  (p  by  -  (j) 


Note  that  a  similar  lemma  is  true  for  DFR  distributions  with  all  in¬ 
equalities  reversed.  From  Lemma  2.  7  we  obtain  an  immediate  compari¬ 
son  between  the  moments  of  an  IFR  distribution  and  the  corresponding 
moments  of  exponential  distribution  with  the  same  mean. 


th 


If  F  i_s  IFR(DFR)  with  r  moment  juLr,  then 


<_Rr  +  1)  M-] 
(>)  1 


2.  +  n/s 

to  1 


Proof. 


Choose 


x 


r  - 


1 


r  y  1 


0  <  r  <  I . 


in  Lemma  Z.  7.  Then 
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I'r 


roo  _  r 00  _ 

■  I  xr  "  *  F(x)  dx<  r  I  xr~lG  (x)dx.  =  P(r  + 


for  rl>l.  ForO«C.  r<l,  ($(x)  =  xr*!  is  decreasing,  so  that  the  inequal¬ 
ity  is  reversed. 

2 

In  particular,  for  an  IFR  distribution  jJL £<!2jx  ^  no  that  the  variance 
|Li|i  and  so  the  coefficient  of  variation  OV^i,j<Tl.  The  inequalities 
are  reversed  for  DFR  distribution! 


Lemma  2.  7  can  also  be  used  to  show  that  the  mean  life  of  a  series 
system  with  IFR  components  whose  means  are  f j,-[i  =  1,  2,  .  .  .  ,  n) 
exceeds  the  mean  life  of  a  series  system  with  exponential  components 
and  means  (i  =  1,  2,  .  .  .  ,  n).  Just  the  reverse  is  true  for  a  parallel 
system. 

_  “x/u. 

Theorem  2.  9.  If  F^x)  is  IFR(DFR)  with  mean  ji_  and  C^(x)  =  e 
(i  =  1 ,  2,  .  .  .  ,  n),  then 


r°°_n 

L 


F.(x)dx  > 

1  «) 


G(x)dx=l/£  l//.i.  , 
i-1  '  1 
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(or  l<Ii<Jn.  By  recureian  we  obtain 


'‘I*? 


(b)  The  proof  is  similar, 

Upper  and  lower  bound*  on  F(t)  when  F  i*  IFR  and  |X  ■  1 ,  H  are 

given  have  been  tabulated  by  Barlow  and  Manhall  ( I9b3b),  Short  tablet 
are  reproduced  h#re  as  Tables  II  and  III  respectively,  Table  IV i  repro¬ 
duced  from  Barlow  and  Marshall  (1983b),  tabulates  the  lower  bound  on 
F(t)  whon  F  it  DFR  and  M<1  ■  1<  ■  ^  “  l(  »1)4, 

Many  additional  bounds  have  bean  obtained  on  the  failure  rata  r(x) 
itself  and  on  the  density  f(x)  by  Bsrlow  and  Marshall  and  will  bs  presented 
in  a  forthcoming  report  by  them, 

We  aloes  this  section  by  presenting  bounds  concerning  renewal  pro¬ 
cesses  based  on  underlying  IFF(DFR)  distributions,  First  we  give  a 
lower  bound  bn  M(t),  the  expected  ..umber  of  renewals  in  [o,  t)  true  for 
all  renewal  processes,  and  an  upper  (lower)  bound  on  M(t)  when  the  under¬ 
lying  distribution  of  the  renewal  process  is  IFR(DFR),  The  proof  may  be 
founC  in  Barlow  and  Pvoschan  1 1983), 

Theorem  Z.  jo.  (i)  M(t)  >t >1  F(x)dx  -  1  >t/ at.  •  1 , 

1  “Jo  ~  1  1 

(il)  If  F  U  IFR(DFR),  then 


for  all  tf>  0. 


TABLE  II 


UPPER  BOUNDS  ON  1  -  r(t) 

N  •  . 

(?  i*  IFR,  ■  !  tdP(t)  «  1,  H9  ■  J  t*dF(t)) 

1  o  8  0 

h  1.1  1.3  1.3  1.4  1.5  1.6  1.7  l.s  1.9  a.o 

t 

0.1  1.000  1.000  1.000  1.000  1.000  1.000  1.000  1.000  0.953  0.905 

0.2  1.000  1.000  1.000  1,000  1.000  1.000  0.965  0.911  0.062  0,019 

0.3  1.000  1.000  1.000  1.000  0.994  0.929  0.873  0.824  0.780  0.741 

0.4  1.000  1.000  1.000  0.970  0.901  0.84 k  0.791  0.746  0.706  0.671 

0.5  1.000  1.000  0.957  0.881  0.817  0.763  0.716  0.675  0.639  0.607 

0.6  1,000  0.959  0.871  0.800  0.741  0.691  0.648  0.611  0.578  0.549 

0.7  0.988  0.878  0.794  0.727  0.672  0.626  0.587  0.553  0.523  0.497 

0.0  0.914.  0.805  0.725  0.661  0,610  0.568  0.532  0.301  '  0.474  0.450 

0.9  0.850  0.740  0.662  0.602  0.554  0.515  0.482  0.453  0.429  0.407 

1.0  0.797  0.603  0.605  0.540  0.503  0.467  0.436  0.410  0.300  0.360 

8 

1.1  0.756  0.632  0.555  0.499  0.457  0.423  0.395  0.371  0.351  0.333 

1.2  0.633  0.589  0.509  0.456  0.U5  0,384  0.358  0.336  0.318  0.302 

1.3  0.411  0.555  0.469  0.416  0.378  0.348  0.324  0.304  0.288  0.273 

1.4  0.259  0.444  0.434  0.381  0.344  0.316  0.294  0.276  0.260  0.247 

1.5  0.163  0.321  0.405  0.349  0.313  0.207  0.266  0.250  0.235  0.224 

1.6  0.105  6.231  0.332  0.321  0.285  0.260  0.241  0.226  0.213  0.202 

1.7  0.068  0.167  0.254  0.296  0.26P  0.237  0.219  0.205  0.193  0.183 

1.6  0.046  0.122  0,194  0.259  0.238  0.215  0,198  0.185  0.175  0.164 

0.031  0.089  0.149  0.205  0.210  0.196  0.100  0.160  0,150  0.150 

2.0  0.021  0.066  O.lli  0.163  0.201  0.178  0.16?  0.152  0.143  0.136 

2.1  0.015  0.050  0.090  O.iiO  3.169  0.162  0.140  0.130  0.130  0.123 

2.:?.  0.010  0.038  0,070  0.105  0,138  0.148  0.134  0.125  0.U7  0-111 

2.3  0.007  0.029  0.056  0.084  0.114  0.135  0.122  0.113  0.106  0.10* 

2.4  0.005  0.022  0.044  0.068  0.094  0.119  0.111  0.102  0.096  0.PJ1 

2.5  0.004  0.017  0.035  0.056  0.077  0.099  0.101  0.093  0.087  0.083 

2.6  0,003  0.013  0.028  0,046  0.064  0.003  0.092  0.084  0.079  01,75 

2.7  0.002  0.010  0.023  0.037  0.053  0.070  0.004  0.076  0.07i  0.060 

2.8  0.003  0.000  0.010  0.031  0.045  0.059  0.074  0.069  0.065  0.061 

2.9  0.001  0.006  0.015  0.026  0.037  0,050  0.063  0.063  0.059  0.055 

3.0  0.001  0.005  0.012  0.021  0.Q31  0.043  0.054  0.057  0.053  0.050 


TABLE  HI 


LOWER  BOUNDS  ON  1  -  F(t) 


W  M 

(F  1b  XFR,  H.  "  f  rn  1,  n  .  {  t2dK(t)) 

1  0  0 


t 

**2 

Z  1.1 

1.2 

1,3 

1,4 

1.5 

1.6 

1.7 

1.8 

1.9 

0.1 

0.974 

0.955 

0.941 

0.930 

0.922 

0.916 

0,912 

0.900 

0.906 

0.2 

0.949 

0.913 

0,006 

0.066 

0.051 

0.040 

0,031 

0.825 

0.021 

0.3 

0.925 

0.072 

0.034 

0.006 

0.705 

0.770 

0.750 

0.750 

0.744 

0.4 

0.900 

0.033 

0.705 

0.750 

0.724 

0.705 

0.691 

0.681 

0.674 

0.5 

0.060 

0.709 

0.736 

0.690 

0.660 

0*646 

0,630 

0.619 

0.611 

0.6 

0.019 

0.731 

0.677 

0.640 

0.613 

0.592 

0.575 

0.562 

0.553 

0.7 

0.747 

0.655 

0,605 

0.573 

0,551 

0.534 

0.521 

0.510 

0.501 

0.0 

0.640 

0.561 

0.524 

0.501 

0.406 

0.475 

0.466 

0.459 

0.453 

0.9 

0.501 

0.459 

0.441 

0.430 

0.423 

0.410 

0.414 

0.411 

0.400 

1.0 

0.367 

0,367 

0.367 

0.367 

0.367 

0.367 

0.367 

0.367 

0.367 

1.1 

0.269 

0.294 

0.306 

0.314 

0.319 

0.323 

0.326 

0.329 

0.3)0 

1.2 

0.000 

0.235 

0.255 

0.260 

0.277 

0.204 

0.209 

0.294 

0.297 

1.3 

0.000 

0.100 

0.213 

0.229 

0.240 

0.249 

0.257 

0,263 

0.267 

1.4 

0.000 

0.000 

0.177 

0.195 

0.209 

0.219 

0.228 

0.235 

0.240 

l.S 

0.000 

0,000 

0.145 

0.166 

0.101 

0.193 

0.202 

0,210 

0,215 

1.6 

0.000 

0.000 

0.000 

0.141 

0.157 

0.169 

0.179 

0.180 

0.3,93 

1.7 

0.000 

0,000 

0.000 

0.117 

0.136 

0.149 

O.j.59 

0.168 

0.174 

1.0 

0.000 

0.000 

0.000 

0.000 

0.116 

0.130 

0.141 

0.150 

0.156 

i.9 

0.000 

0,000 

0.000 

0.000 

0.090 

0.114 

0.123 

0.114 

0,140 

2.0 

0.000 

o.nno 

c.ooo 

0.000 

0.078 

0.090 

0.110 

0.120 

0.126 

2.1 

0.000 

0.000 

0.000 

0.000 

0.000 

0.084 

0.097 

0.107 

0.113 

2.2 

0.000 

0.000 

0.000 

0.000 

0.000 

0.071 

0.005 

0.095 

C.'.O? 

1.3 

r.ooo 

0.000 

0.000 

0.000 

0.000 

0.057 

0.074 

0.084 

0.091 

2.4 

0.000 

0.000 

0,000 

0.000 

0.000 

0.000 

0.044 

0.073 

0,902 

2.5 

0.000 

0,000 

0.000 

0.000 

0.000 

0.000 

0.055 

0.066 

0.074 

2.6 

0.000 

0.000 

0.000 

0.000 

o.coo 

0.000 

0.046 

0.050 

0.066 

2.7 

0.000 

0,000 

0.000 

0.000 

0.000 

0.000 

0.037 

0.051 

0.059 

2.0 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.045 

0.053 

2.9 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.039 

0.040 

3.0 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.033 

0.042 

TABLE  IV 


LOWER  BOUNDS  ON  1  -  F(t) 


«  •  . 

(F  1*  ora,  H,  ■  K  t4F(t)  «  1,  |i,  -  a  rdF(t)) 
1  0  *  0 


“a 


t 

2.1 

2.2 

2.3 

2.4 

2.5 

8.6 

2.7 

2.8 

2.9 

3.0 

0.0 

0.952 

0.909 

0.869 

0.833 

0.800 

0.769 

0.740 

0.714 

0.689 

0.666 

0.1 

0.861 

0.822 

0.787 

0.734 

0.724 

0.696 

0,671 

0.647 

0.623 

0.604 

o.a 

0.779 

0.744 

0.712 

0.603 

0.656, 

0.631 

0.608 

0.586 

0.566 

0.548 

0.3 

0.703 

0.673 

0.644 

0.618 

0.594 

0.571 

0.551 

0.531 

0.514 

0.497 

0.4 

0.638 

0.609 

0.583 

0.559 

0.538 

0.317 

0.499 

0.482 

0.466 

0.451 

0.9 

0.377 

0.351 

0.528 

0.506 

0.487 

0.469 

0.432 

0.437 

0.422 

0.409 

0.6 

0.582 

0.499 

0.478 

0.458 

0.441 

0.425 

0.410 

0.396 

0.383 

0.‘J71 

0.7 

0.473 

0.452 

0.432 

0.415 

0.399 

0.385 

0.371 

0-359 

0.347 

0.336 

0.8 

0.428 

0.409 

0.391 

0.376 

0.362 

0.348 

0.336 

0.325 

0.315 

0.303 

0.7 

0.387 

0.370 

0.354 

0.340 

0.327 

0.316 

0.303 

0.895 

0.286 

0.277 

1.0 

0.350 

0.333 

0.321 

0.308 

0.297 

0.286 

0.276 

0.267 

0.239 

0.231 

1.1 

0.317 

0.303 

0.290 

0.279 

0.269 

0.259 

0.250 

0.242 

0.233 

0.228 

T.a 

0.287 

0.274 

0.263 

0.253 

0.243 

0.233 

0.227 

0.220 

0.213 

0.207 

A. 3 

0.239 

0.248 

0.238 

0.229 

0.220 

0.213 

0.206 

0.199 

0.193 

0.188 

1.4 

0.239 

0.224 

0.215 

0.207 

0.200 

0.193 

0.186 

0.181 

0.173 

0.170 

1.5 

0.212 

0.203 

0.195 

0.187 

0.181 

0.173' 

0.169 

0.164 

0.159 

<1.153 

1.6 

0.192 

0.184 

0.176 

0.170 

0.164 

0.158 

0.153 

0.149 

0.144 

0.140 

1.7 

0.174 

0.166 

0.160 

0.154 

0.148 

0.143 

0.139 

0.135 

0.131 

0.127 

1.0 

0.157 

0.15C 

0.144 

0.139 

0.134 

0.130 

0.126 

u.122 

0.119 

0.116 

1.9 

0.142 

0.136 

0.131 

0.126 

0.122 

0.118 

0.114 

0.111 

0.108 

0.103 

8.0 

0.129 

0.123 

0.138 

0.114 

0.110 

0.107 

0.104 

0.101 

0.098 

0.095 

0.1 16 

0.111 

0.107 

0.103 

0.100 

0.097 

0.094 

0.091 

0.0*9 

0.08/ 

2.2 

0.103 

0.101 

0.097 

0.094 

0.090 

0.088 

n.oej 

0.083 

0.081 

0.079 

4.3 

0.095 

0.091 

0.088 

0.085 

0.082 

0.079 

0.077 

0.073 

0.073 

0.071 

2.4 

0.086 

0.083 

0.079 

0.077 

0.074 

O.CflW 

0;070 

0.068 

0.066 

0.065 

2  5 

0.078 

0.075 

0.072 

0.069 

0.067 

0.065 

0.063 

0.062 

0.060 

0039 

2.6 

0.070 

0.068 

0.065 

0.063 

0.061 

0,059 

0.057 

0.056 

0.053 

0.053 

2.7 

0.064 

0.061 

0.059 

0.057 

0.055 

0.054 

0.052 

0.051 

0.050 

0.048 

2.8 

0.058 

0.055 

0.053 

0.052 

0.050 

0.049 

0.047 

0.046 

0.045 

0.044 

2.9 

0.052 

0.050 

0.0415 

0.047 

0.045 

0.044 

0.043 

0.042 

0.041 

0.040 

3.0 

0.047 

0.045 

0.044 

0.042 

0.041 

0.040 

0.039 

0.038 

0.037 

0.036 

TABLE  IV  (continued) 
LOWER  BOUNDS  ON  1  -  F(t) 
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v  Ib 

(f  i*  DPR,  |».  »  J  tdf(t)  «  1,  it,  ■  J  t*dF(t)) 
1  0  z  0 


*2 


t 

3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.8 

3.9 

4.0 

0.0 

0.645 

0.625 

0.606 

0.588 

0.571 

0.555 

0,540 

0.526 

0.512 

0.500 

0.1 

0.583 

0.567 

0.550 

0.534 

0.519 

0.504 

0.491 

0.478 

0.466 

0.455 

0.2 

0.331 

0.514 

0.499 

0.485 

0.471 

0.458 

0.446 

0.435 

0.424 

0.414 

0.3 

0.481 

0.467 

0.433 

0.440 

0.428 

0.417 

0.406 

0.396 

0,386 

0.377 

0.4 

0.437 

0.424 

0.4.U 

0.400 

0.389 

0.379 

0.369 

0.360 

0.351 

0.343 

0.5 

0.396 

0.365 

0.374 

0.363 

0.354 

0.344 

0.336 

0.327 

0,320 

0.312 

0.6 

0.360 

0.349 

0.339 

0.330 

0.321 

0.313 

0.305 

0.298 

0,291 

0.284 

0.7 

0.326 

0.317 

0.308 

0.300 

0.292 

0.285 

0,278 

0.271 

0.265 

0.259 

0.6 

0.296 

0.286 

0.280 

0.272 

0.265 

0.259 

0  253 

0.247 

0.241 

0.236 

0.9 

0.269 

0.261 

0.254 

0.248 

0.2U 

0.235 

0.230 

0.224 

0.219 

0.215 

1.0 

0.244 

0.237 

0.231 

0.225 

0.219 

0.214 

0.209 

0.204 

Ok  200 

0.195 

1.1 

0.222 

0.215 

0.210 

0.204 

0.199 

0.195 

0.190 

0.186 

0.182 

0.178 

1.2 

0.201 

0.196 

0.191 

0.186 

0.181 

0.177 

0,173 

0.169 

0.166 

0.162 

1.3 

0.183 

0.178 

0.173 

0.169 

0,165 

0.161 

0.157 

0.194 

0.151 

0.148 

1.4 

9.166 

0.161 

0.137 

0.154 

0.150 

0.147 

0.14? 

0.140 

0.137. 

0.135 

1.3 

0.151 

0.147 

0.143 

0.140 

0.136 

0.133 

0.130 

0.128 

0.125 

0.123 

1.6 

0.137 

0.133 

0.130 

0,127 

0.124 

0.121 

0.119 

0116 

0.114 

0.112 

1.7 

0.124 

M21 

O.xltt 

0.115 

0.113 

0.110 

C.luf 

0.106 

0.104 

0,102 

1.8 

O.i  13 

0.110 

O.lt/7 

0.105 

0.103 

0.100 

0.098 

0.096 

0.095 

0.093 

1.9 

0.102 

0.100 

0.098 

0.093 

0.093 

0.091 

0.090 

0.088 

0.086 

0.085 

2.0 

0.W3 

0.091 

0.089 

0.087 

0.085 

0.083 

0.082 

O.Otfo 

2.0"9 

0.077 

2.1 

0.064 

0.082 

0.081 

0.079 

0.077 

0.076 

"-074 

0.073 

0. 072 

0.C70 

2.2 

0.077 

0.073 

0.073 

0.C72 

0.070 

0.069 

0.068 

0,066 

0.065 

0.764 

2.3 

0.C70 

0.066 

0.067 

0.065 

0,064 

0.063 

0.062 

0.060 

O.C59 

0.058 

2.4 

0.063 

0.062 

0.061 

0.059 

0.058 

0.057 

0.056 

0.055 

0.05'. 

0.053 

2.5 

0.057 

0.056 

0.055 

0.054 

0.053 

0.052 

0.051 

0.050 

0.049 

0.048 

2.6 

0.052 

0.051 

0.050 

0.049 

0.048 

0,047 

0, 046 

0.046 

0,045 

0.044 

2.7 

0.047 

0.046 

0.045 

0.043 

0.044 

0.043 

0.042 

0.042 

0.041 

0.040 

2.6 

0.043 

0.042 

0.041 

0.041 

0.040 

0.039 

0.038 

0.038 

0.037 

0,037 

2.9 

0.039 

0.038 

0.038 

0.037 

0.036 

0.036 

0.035 

0.034 

0.034 

0.033 

3.0 

0.036 

0.035 

0.034 

0.034 

0.033 

0.032 

0.032 

0,031 

0.031 

0.030 
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Note  that  as  a  const  quince,  for  an  underlying  1FR  distribution  tha 
expected  number  of  renewals  is  known  within  an  arror  of  1/2  for  each 
moment  of  time.  Note  alio  that  in  tha  DFR  eaaa.  tha  inequality  (it)  is  an 
improvement  over  inequality  (!)  ainee 


tF(t )l  f  F{x)dx  >  f  T(x)dx//*F(x)dx  a  t /[  F(x)dx  -  1. 
Jh  *'n 


Next  we  compare  momenta  of  N(t),  tha  number  of  renewals  in[0,t  j 
whan  tha  underlying  distribution  F  is  IFR(DFR)  with  tha  corresponding 
moments  of  a  Poisson  process. 


Theorem  2.  il .  1£  F  is  IFR(DFR)  with  mean  ju,.,  then 

-t/fij 

u  vs,  ik(t/Kl),e 

(a)  EN**(t)  ^  £  - H - • 

{>)  J-0 


(b)  Var  N(t)<  EN(t)<  t/  u... 

(>)  ri 

See  Barlow  and  Proschan  (19b3)  for  a  proof. 


3.  QUALITATIVE  CONCLUSIONa  ASSUMING  UNDER LY1NO  DISTRI¬ 
BUTION  IS  IFR^DFR),  In  a  number  of  reliability  models •  tli.-  assumption 
that  the  underlying  failure  distributions  are  IFR(DFR)  yialds  useful 
Hualitailvu  relationships,  and  in  optimisation  problems,  helpful  infor¬ 
mation  about  the  form  of  the  solution.  In  this  section  we  shall  present  a 
selection  of  reliability  models  that  Illustrate  bow  the  assumption  of 
underlying  IFR(DFK)  distributions  simplifies  ths  solution. 


3.  1  Allocation  of  Spares  undsv  Constraints.  A  system  is  required 
to  operate  for  the  period  Jjb,  tjj]  .  When  a  component  falls,  It  is  Immadl- 
utely  replaced  by  a  spare  component  of  the  eame  type  if  available.  If  no 
■pare  It  available,  eystem  failure  results.  Only  the  spares  originally 
provided  may  be  used  for  replacements;  1.  e.  ,  no  resupply  of  spares  can 
occur  during  [o,  t0]  .  The  cost  of  a  single  epare  of  the  l-*1  type  is  ct, 
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i  *  1,  ,  ,  ,  ,  kj  the  total  co it  for  spare*  i*  c(n)  ■  jjT  c^,  wh' re  n,  ■ 

(nj ,  .  .  .  i  n^)i  and  nl  ■  number  of  (pare*  of  type  l  Initially  provide*, 
i  »  1 . k. 

Wc  now  describe  the  original  lyitem  before  any  replacement*  are 
made.  The  lyitem  contl  *t*  of  d{  "positions"  or  "sockets"  each  filled 
by  a  cemponentof  type  1,  1  »  1 ,  l,  ,  .  . ,  k.  The  variou*  component* 
of  a  given  type  may  be  u**d  at  different  level*  of  intensity  and  may  be 
subject  to  different  environmental  stresses,  so  that  for  full  generality 
we  assume  the  life  of  the  Ith  component  of  type  l  (occupying  position 
l,  j,  aay)  ha*  probability  distribution  7^,  |  ■  1 . dj,  1  ■  1 ,  .  ,  ,  ,  k, 

Each  replacement  ha*  th*  lame  life  diatribition  aa  ita  predecessor; 
component  live*  aro  mutually  independent.  Position  i,  )  i*  not  required 
to  be  in  constant  operation  throughout  fo,  tp]  ,  but  rather  la  scheduled 

to  operate  for  a  period  of  duration  ty<t0,  j  ■  1 . dj,  i  ■  1 ,  ,  .  . ,  k, 

Thl*  model  la  discussed  in  detail  in  Proschan  (I960). 

We  seek  a  family  of  undomlnswd  spare*  allocations,  A  spares  allo¬ 
cation  n  i ■  undominated  if  R(n')  >  R(n)  implies  c(n')>c(n)  while 
P(n')  "  R(n)  Inipflcs  c(n1^c(njT  where  n1  U  any  other  spa’res  allocation, 
and  R(n)  (called  system  reliability)  is  the  probability  of  no  system  shut" 
down  during  ^0,  tQ]  resulting  from  shortage  uf  spar**,  That  is,  a  spares 
allocation  is  undominated  if  for  its  cost  it  achieves  maximum  reliability. 

A  family  of  un  dominated  allocations  represent*  a  tremendouu  reduction 
of  possibilities  that  the  decision  maker  needs  to  consider  in  arriving  at 
an  ordinal  spa  roc  allocation,  We  shall  develop  a  method  tor  generating 
AU.tccBBivcly  larger  undominated  spares  allocations  which  dapsndi  on 
Xn  R(_n  )  being  concave.  Wa  shall  see  below  that  J. n  R(n)  is  cc~-ave 
when  each  of  the  Fjj  is  I7R. 

First  note  that  since  shortage  in  any  position  result*  in  system 
fit  llurc 
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where  Rj(m)  l>  the  probability  of  no  ehortege  of  unite  of  type  l 
assuming  m  spares  of  typs  1  ere  Initially  provided,  We  can  convert 
tne  objective  function  R(n)  being  maximised  Into  a  eum  of  ttrmi  each 
depending  on  a  elngle  unknown  only  by  taking  logarithmst 

(3.  3)  k 

A  n  R(n)  ■  £  i.n  Rj(ni). 
t»l 

Note  that  maximising  in  R(n)  i»  equivalent  to  maximising  R(n)  since 
■tn  x  tea  monotone  increasing  function  of  x. 

Next  w*  shall  describe  a  procedure  for  generating  a  family  of  undom* 
Innted  allocations  which  Is  Intuitively  quite  reasonable.  The  underlying 
Idea  Is  that  we  will  construct  successively  larger  spares  allocations  by 
adding  one  spare  at  a  time)  the  spar*  we  add  will  bs  the  one  which  pro* 
vldes  greatest  Improvement  In  system  reliability  per  dollar  spent. 


Procedure.  Start  with  the  cheapest  allocation  (0,  0 . 0),  Obtain 

successively  more  expensive  allocations  as  follows.  If  the  present  allo» 
cation  Is  n.  determine  the  Index,  call  it  Jq.  for  which 


~^n  R^  ♦  1)  -  <n  Ri(uft)} 


■.«  maximum  over  1*1,  2, 


type  to  n  to  obUin  the  newt  larger  allocation  (nj, 


k.  Then  add  a  elngle  unit  of  the  1q 


th 


l0 


+1 1 


nJ. 


*o-> 


Note  thet  adding  the  m  ost  to  ~ln  R(n)  per  dollar  (pent  <»  equivalent 
to  multiplying  R(n)  by  the  largeet  factor  poeslble  per  dollar  spent.  V/« 
shall  see  In  Theo  rem  3.  1  below  that  the  Procedure  generate*  only  un- 
domlnated  allocations  if  <  n  R(n)  la  concave. 


Theorem  3,  l.  It  in  R(n)  l*  concave,  then  each  eparee  allocation 
generated  by  the  Procedure  is  undomlnated. 


Proof,  Let  n*  be  generated  by  the  Procedure,  ig  denote  th*  Index 
of  the  laet  component  type  added  In  arriving  at  n*by  the  Procedure,  and 
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Let  n  he  any  other  allocation  such  that  R{n)>R(n*).  Designate  the  set 

of  indie-  »  tor  which  by  I  and  the  set  of  indices  for  which  'u^n?' 

i  i  '  1  1  i 

by  I.,.  Then 


since  for  t  in  ”,  and  h  •  1,  2.  ,  .  , ,  An  +  h)  ■  /«  l^in**  h  - 

while  for  i  in  Ij  and  h  ■  0»  1 1  St  •  ■  •  •  4ii  Rjjn|  *  h)  *  »^i\  R^(rt|  *  h  .  l^r 
Thee:.  *?*  consequences  of  4n  R|jj)  feting  concave,  i,  e,  .  of  A(n  R^n)  s 
4.*  +  1)  •  4n  R^n)  deer  sating  in  n.  Thus 

0<&  °‘"t  vt  ■ 

In  c  similar  fashion,  assuming  R(rj  *  R{jfi  ws  may  prove  that 
k  * 

Z  vi  £  E  • 

Thus  e*  is  undominatsd,  |J 

Note  that  Theorem  3,  1  requires  that  in  «(r?  be  concave,  It  turns 
out  that  in  R(n)  is  not  concave  for  all  componen” failure  distributions 
Fy,  but  as  shown  in  Thsorem  3.  2  btlow,  la  ooncavs  If  each  Fy  is  3PR . 

(As  usual,  the  very  natural  IFR  assumption  simplifies  the  solution 
considerably  ) 
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First  we  must  establish  the  relationship  between  system  reliability 
R(n)  and  the  F^.  The  lives  of  the  component  in  position  1,  |  and  its 
successive  replacements  constitute  a  renewal  process  (as  long  as  spares 
are  available).  Lft  Nn  ■  number  of  failures  in  position  i,  )  during 
[O.tjj],  Then  P{NU  ■  n]  ■  F}^  (tjj)  -  f[^)  (t^),  Also  Rt(n),  the 

probability  that  no  shortage  of  spares  of  type  l  occurs  during  [Oi(q] 
assuming  n  spares  of  type  i  are  stocked,  is  glvon  by 

R^n)  -  p[nu  +  N,?  +  .  .  ■  +  Nj nj  . 

or  explicitly,  by 

(3,4)  dj 

Rl(n).  £  IT  PLNi|"n)J' 

«l+ <  •  •  +  ndiShlBl 

Now  we  may  state 

Theorem  3,  Z,  If  each  F^,  l_sIFR,  then  In  R(n)  Is  concave.  The 
proof  may  be  found  in  Barluw  and  Prosehan  ( 1954),  Chapter  VI, 

Exponential  Failure  Distributions,  Suppose  each  failure  density  Is 
exponential!  specifically,  suppose 


»-t/p 


£l|  M 


It  follow  i  that 

PK 


n]  =  ' 


)v.r  t  JO 
for  t<0. 


iiL 

a  Poisson  frequency  function  with  parameter  ^  (Arrow,  Karlin,  Scarf 
(1958),  page  27?,).  Hence 
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(3.  5) 

R  (n) 

-M'i 

=  e 

where 

1 

Mb  = 

E.TCt 

j=i 

i=o 


since  the  convolution  of  Poisson  frequency  functions  is  a  Poisson  fre¬ 
quency  function  with  parameter  given  by  the  sum  of  the  separate  para¬ 
meters  (Cramer  (1946),  page  205.  )  . 

Using  (3.  5),  it  is  a  relatively  simple  matter  to  apply  the  Procedure 
above  for  generating  undominated  allocations  in  the  present  case  of 
underlying  exponential  failure  densities.  See  Proschan  (I960)  or  Barlow 
and  Proschan  (1964),  Chapter  VI,  fcr  worked  examples. 

3.  2  Comparison  of  Age  and  Block  Replacement.  Among  the  most 
useful  replacement  policies  currently  in  popular  use  are  the  age  replace¬ 
ment  policy  and  the  block  replacement  policy.  Under  an  age  replacement 
policy  a  unit  is  replaced  upon  failure  or  at  age  T,  a  specified  positive 
constant,  whichever  comes  first.  Under  a  block  replacement  policy  a 

unit  is  replaced  upon  failure  and  at  times  T,  2T,  3T . We  assume 

t.ict  units  fail  permanently,  independently,  and  that  the  time  required  to 
perform  replacement  is  negligibly  small.  Block  replacement  is  easier 
to  administer  since  the  planned  replacements  occur  at  regular  intervals 
and  fv  are  readily  scheduled.  This  type  of  policy  is  commonly  used 
with  digital  computers  and  other  complex  electronic  systems.  On  the 
other  hand,  age  replacement  seems  more  flexible  since  under  this 
policy  planned  replacement  takes  into  account  the  age  of  the  unit.  Tt  is 
therefore  of  some  interest  to  compare  these  two  policies  with  respect 
to  th  number  of  failures,  number  of  planned  replacements,  and  number 
of  removals.  ("Removal"  refers  to  both  failure  replacement  and  planned 
replacement.  ) 

oiock  replacement  policies  have  been  investigated  by  E.  L.  Welker 
(1959),  R.  F.  Drenick  (I960),  and  B.  J.  Flehinger  (1962).  Age  replace¬ 
ment  policies  have  been  studied  by  G.  Weiss  (1956)  and  Barlow  and  Pro-- 
schan  (1962)  among  others.  The  results  presented  below  are  based  on 
Barlow  and  Froschan  (l  963). 
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We  shall  compare  block  replacement  with  age  replacement,  both  using 
replacement  interval  T.  Ibr  example,  block  replacement  le  more  wasteful 
since,  as  we  shall  show,  more  unfalled  components  are  removed  than  under 
a  policy  based  on  age,  Likewise,  the  total  number  of  removals  for  both 
failed  and  unfailed  components  is  greater.  However,  ae  one  would  euspect, 
under  die  1FR  assumption  the  expected  number  of  failures  wi,  11  be  leei 
under  block  replacement,  Finally,  exactly  [t/T ]  planned  replacements  will 
be  made  in  [o,  tI  under  block  replacement,  whila  no  more  than  f  t/T  J  can 
be  made  under  age  replacement.  L  J 

We  ehall  denote  the  number  of  removal*  in  [o,  T  j under  a  block  policy 
by  Ng(t)  and  the  number  of  removal*  in  |0,  Tj  under  an  age  policy  by 
NA(t).  As  we  show  in  Theorem  3,  3,  Ng(t)  1*  stochastically  larger  than 
NA(t). 

Theorem  3.  3,  p[NA(t>  >n]<p[NB(t)  >n]  for  n  ■  0,  1,  l . 

Proof,  Let  {XjJ  denote  a  realisation  of  the  lives  of  successive  corn* 
k»l 

ponents.  We  shall  compute  what  would  jaave  occurred  under  an  age  and  under 
a  block  replacement  policy.  Let  TA(T^  denote  the  time  of  the  nth  removal 
under  an  sge  (block)  replacement  policy,  Then 

T“  *min(Tf'1  +  T,  T1,}"  *  +X  ) 

A  n  An 

Tg  *  min  (Tp"‘+  ,  Tg'1  +  XJ 

whore  0(,(0<5(<T)  i#  ihu  remslning  life  to  a  scheduled  replace,','  '•tit. 

Since  Initially  T^  ■  Tg  ,  we  have  by  Induction  T^  >Tg  .  Thus  for  any 
realisation  {x^}  ,  NA(t)  is  smaller  than  Ng(t).|| 

Next  we  shall  use  Theorem  3.  3  to  establish  a  lower  bound  on  the  ie» 
m.  wai  function  quite  independent  of  replacement.  Let  NJ(t)  (Nj,  (t)) 

denote  the  number  of  failures  in  [o,t]  under  age  (block)  replacement  at 
interval  T,  Theorem  3.4  below  shows  thnt  the  number  of  failures  per 
unit  of  time  under  block  replacement  at  interval  T  is,  in  the  limit,  — L( 

where  M(T)  is  the  renewal  function  E  F(k>(T), 

k*l 
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(3. 6)  Nj|(t)  MjT) 

Thtoum  3.  4.  Um  - . .  «  -■i""  a.  . 

Proof,  Let  N+^(t)  denote  the  numbir  of  failures  in  [(l-l)T,  iT ]  . 
Clearly  the  random  variables  N^jT)  arc  independent,  identically  distri¬ 
buted,  and  for  kT<t<(k+l)T, 


kN+(T)  N*(t>  kfl  N*(T) 

S-2L-  <_■_  s  t  _2i_ 

TkTlJiL  t  l  (k+DT- 


r  (k+i)xr^_ , 


Letting  t-*ea,  we  have  and 

t~*o  x  T 

by  the  strong  law  of  large  numbers,  || 
Prom  (3.  7)  we  see  diet  also 


um  SNft^  .  mi 

t~»w  t  ^ 


Since  the  numoer  ot  removals  v.i.h  block  rspla^ement  < s  stochastically 
greater  than  the  number  of  rentals  with  age  replacement  by  Theorem  3,  3, 
we  *e*  that 

<:’'9>  Enb<4>  >l.mBNA(‘) 


But  using  13.  B) 


„  enbu)  M(T)  1 

im  +T 
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end 


l<m 


■Na(») 

“*"t 


by  the  elementary  renewal  theorom.  since  for  age  replacement  the  times 
between  removal*  constitute  a  renewal  process.  Substituting  In  (3,  9)  we 
have 

+  i — 

T  ■  T  c  T 

/  ?(x)dx 
*'0 


or 

(3. 10) 


M(T)  * 


F  (x)  dx 


•1. 


W*  thus  have  the  following  lower  bound  on  the  renewal  function,  re* 
grrdless  of  the  underlying  distribution, 

Theorem  3.  5,  M(t)  >  - S - -  >  77  -  l  for  «  <  t  <*o.  (3-  111 

-  -  T_  p 

I  r  (x)  dx 
Jo 


Proof.  The  first  inequality  has  already  been  established  In  (9.  >0). 


The  second  follows  from 


jT  *  F  (x) 


Tlx)dx  - 


Next  we  shall  compare  the  number  of  failures  undsr  the  two  policies 
and  as  a  conssquenoe  obtain  an  upper  bound  or>  the  renewal  function  valid 
when  the  underlying  failure  distribution  is  IFR.  First  ws  obtain  ths  long 
run  average  time  between  falluree  under  a  block  rsplacument  policy. 
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Let  denote  '.he  succeseive  times  between  failures  under  a  block 

replacement  pulley  having  replacement  interval  T. 


Theorem  3.  6,  lim 
t-*ee 


YliY2+,  .  .  +VN<ut) 


T$TT 


Proof.  Note  that 


Letting  t-+» and  applying  Theorem  3,1  we  have  the  dealred  result.  || 

Theorem  3.  7  below  shows  that  the  number  of  failures  under  an  age 
replacement  policy  is  stochastically  greater  than  the  number  of  failures 
under  a  block  replacement  policy. 

Theorem  3,7,  If  F  U  IJTR,  then 

pf.NX(t)  >  n]  >  p[n*  (t>  >  nj. 

The  proof  may  be  found  in  barlow  and  Proeehen  (1963). 

IJ  eing  Theorem  3,7  wc  obtain  the  following  upper  bounds  for  tn« 
newel  function  M(t)  when  the  underlying  failure  dietribution  ie  IFR 

Theorem  3.  8,  It  F  le_IFR, 

M!t)  <  -Jllil -  <±  isu  0  <  t  <  ce  . 

F(x)dx 

JO 


5B 
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(unction  rf  T.  (See  Berlov/  end  Proschan  (1964) 


tr(t)  <  ^tre* ) 

f  F(x)dx  f  F(x)«lx 

0  ■'O 


the  second  of  the  deeired  concluelone.  II 

Combining  Theorem*  3.  3  and  3.  8.  we  obtain  very  oloee  bounds- on 
M.(t)  when  the  underlying  failure  distribution  is  IFRi 

(3.15) 

•£--  l<  W)<,4r  *  °<  *<*>• 

p  p 

Thus  for  all  non-negative  values  of  t.  M(t)  may  be  approximated  with 
an  error  of  at  most  1/2. 

4.  PRESERVATION  OF  MONOTONE  FAILURE  RATE.  Next  wo  con¬ 
sider  operation*  under  which  a  monotone  failure  rate  ie  preserved.  For 
exempt*,  what  structure*  have  tho  monotone  failuvu  rate  property  when 
'heir  individual  component*  have  this  property?  Is  a  monotone  failure 
rate  preserved  under  convolution  or  under  mixture  of  distribution*? 

Tl.o  results  presented  in  this  section  are  based  on  Esary  and  Proschan 
(r,">3)  and  Birlow.  Marshall,  and  Proschan  (1963). 

Theorem  4,  1,  lf_  F^  and  F,  arc  IFR,  then  their  convolution  H, 

ilyenby 

H(t)  mj  Fj  (t  -  x)dF2(x)  . 

-  90 

is  also  IFR. 

Proof.  Assume  F  has  density  f  ,  F  has  density  f,  .  For  t.<t  . 
u,<iT~fcrm  1  12  212 

1  2 


Chapter  111. )  Hence 

•  _L 
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O-l  Htti-uj)!  i(  j-1( 2-  |^l(tl-rtl2(.-uJ)d.l  ^  £  |  7l(t1.*k)ll£2(*k-uJ)Uead*1 


by  problem  6b,  page  48,  Pblya  end  See  go  (1928),  Integrating  the  inner 
integral  by  parte,  we  obtain 


Vz 


-91 

«*■ 

a 

■ 

a— 

>— » 

a 

N 

V*1  *  Ul>  *1(,1  ■  u2> 

1\<t2  -  .j)  fj(t2  -  »2> 

f2^2“ul>  *V»2*U2> 

da2da 


1  * 


The  eign  of  the  firat  determinant  1*  the  aim*  aa  that  of 

V*2  *  V  ^‘z  •  #2>  <l(tl  ’  7l(tl  ’  ,2> 

*V*2  '  ^(tj  -  aj)  ^(tj  -  a2)  F^  -  a^ 


»  turning  non-earo  denominator*,  But 


2-  V 
'Az  ‘  *iV  *2 


V* 


by  Itypotheala,  while 


60 


Design  of  Experiment* 


since!  »e  pointed  out  in  Section  1 ,  F^  is  logarithmically  concave.  Thus 
the  first  determinant  is  non  •negative.  A  similar  argument  holds  for  the 
second  determinant,  so  that  D>0.  But  this  implies  H  is  logarithmically 
concave,  and  therefore  K  is  1FR, 

If  F  and/or  G  do  not  have  densities,  the  theorem  may  be  proved 
in  a  similar  fashion  using  limiting  arguments,  II 

It  is  of  Interest  to  note  that  the  DFR  property  l*  not  preserved 
under  convolution.  A  counterexample  is  obtained  if  we  convolute  densities 

fh  -1  -X 

fj(x)  -  f2ix)  •  ,K-pgp-  {ot  *  >°  * 

with  1  / 2<  <X< \ ,  However,  it  is  true  that  a  mixture  of  DFR  distributions 
is  also  DFR,  as  shown  in 

Theorem  4, 2.  F^t)  Ui^R  distribution  in  t  fflTJUUh 
i  »  1 ,  2,  .  ...  at  >0,  1  ■  l ,  2,  . , , ,  and  ^  at  ■  1  (  then 

iel 


oo 

G(t)  ■  £  *iF,W 
i-l  1 

U  a  DFR  distribution, 

Proof,  First  suppose  chat  Fj(t)  has  a  differentiable  density  f^(t) 
Sin<:o  tho  density  of  any  DFR  distribution  must  be  a  decreasing  funcwoi.. 
we  have  by  3chwar*'s  inequality  that 


L 

i-l 


Since 


f.Ul/F^t)  is  decreasing  in  t,  we  must  have 


1/2 


} 


Ft(t)f;U)<  -  [V*)]  L 


4 


4 
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Hence. 

S,*A»g|;^¥*]  Strive]'  • 


thst  Is 

ait)  h1  U)<"[gU)]?' 

where  g  Is  tie  density  of  0,  so  thst  0  it  DPR. 

If  ssch  Pj  does  not  have  s  differ sntisbls  dsnilty,  the  same  result 
may  be  obtained  by  limiting  arguments.  || 

Mixtures  of  MR  distributions  are  not  necessarily  1PR.  For  example, 
a  mixture  of  two  distinct  exponentials  Is  not  IPR  sines  it  is  not  sxponantlal. 
and  by  the  above  theorem  it  is  DPR.  Theorem  4,  2  together  with  the  test 
fbr  DPR  distributions  in  Section  9  may  be  used  to  pick  Up  differeniies  ln  the  ., 
parameters  of  pooled  samples  each  coming  from  an  exponentially  distributed 
population.  See  Proschan  (1963). 

Theorem  4.  1  proves  that  a  system  consisting  of  one  unit  anda.spsre 
has  Increasing  failure  rate  If  the  components  do.  A  a  one  would  expect, 
the  failure  rats  of  the  system  is  everywhere  less  than  the  failure  rats 
cl'  either  component  If  both  components  have  IPR  failure  distributions. 

Theorem  4,  3.  if.  P^  and  P,  el£  IPR  distributions  ylth  tatlUI'e 
EAiSJ  r^(t)  and  r2(t)  fsuectlvslv  and  P  dsnotss  their  yy.wrhitlou  with 

Mltti'f  I»i i  r(t)  ,  then. 


r(t)  <  min 


[*!<»).  r2(t)J. 


Proof.  By  definition 


r 


fj(t-x)  f2(x)dx 

Tit) 


r|t)  *■ 


JV 
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x)f2(x)dx 


r<t)<r,(t)““ - -  Y'  1 

the  equality  being  ulcer  from 

F(t)  •  Fjlt-xld F  (*)  . 

Tl 

Similarly,  r{t)«^  r2(t),  II 

Next  we  shall  obtain  condition*  under  which  an  increacing  failure  rat* 
of  like  component*  implie*  an  incraaiing  failure  rat*  for  tho  system, 
Assume  then  that  the  syrtam  consists  of  independent  like  component*, 
with  each  component  life  distributed  according  to  the  common,  probability 
distribution  Fit).  At  a  given  instant  of  time  t  each  component  ha* 
reliability  p  ■  F(t)t  the  corresponding  system  reliability  will  be  designated 
by  h(p).  Then  w*  may  prove 

Theorem  4.  4,  Assume  a  structure  with  reliability  function  h(p)  , 
with  each  comp'  jt  life  independently  distributed  according  to  distribution 
F  having  dsns),  /  f,  Then 


(a)  JM.  ,  Ph’tP) 
r(t)  h(p) 


P  »  fit) 


where  r( t)  ■  •  component  failure  rate  at  time  t,  and  R(t)  »  system 


failure  rate  at  time  t; 


tjt)"  1*  *n  increasing  function  of  t  if  and  only  if  IT.* 

decree aing  furr.tion  of  p; 

(c)  if  r(t)  1*  an  increasing  function  of  t  and  i*  a  dacr«a»ing 

function  of  p,  then  R(t)  i»  an  increaslnu  function  of  t. 


Result  (c)  gives  a  simple  suificient  condition  on  a  structure  which 
will  pruuorve  a  monotone  failure  rate  when  a  structure  is  constructed  out 
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of  independent  Ilka  components.  We  shell  present  an  important  class  of 
structures  which  satisfy  this  eufflcient  condition, 

To  prove  (a).  1st  S(t)  represent  the  probability  of  structure  eur- 
vlval  past  time  t|  Is.  ,  S(  t)  ■  h(F(t))>  By  definition 


R|*>  ■  ttI" 


.  hliz L 

Mp> 


P  -  F(t) 


•  «»>  -  Ph'tP) 

h(p) 


_  .  jiL 

p  ■  r(t)  F(t) 


so  that 


Kit)  .  ph'(p) 

r(t)  h(p) 


p  ■  r(t> 


a stabli siting  (a), 


To  prove  (b),  simply  note  that  p  ■  F(t)  is  a  decreasing  function  of  t. 


Finally,  (e)  is  an  immediate  consequence  of  (b)  and  the  fact  that 
p  is  a  decreasing  function  of  t,  || 


An  Important  class  of  structures  for  which  the  condition  E-.lE).  is 

h(p) 

Ft  decreasing  function  of  p  are  the  so-called  It  out  cf  n  structures.  A 
out  of  n  structure  is  one  that  functions  if  and  only  if  at  least  It  com¬ 
ponents  function,  To  prove  that  a  k  out  of  n  structure  consisting  of 


n 


Independent  components  has  s  ratio 


decreasing  in  p,  write 


-±i£l 

ph'(p) 


i-t 

1-p 


n-k 

) 


dt  , 


since 


h(p) 


t  <rsplu-p»“-1 

i-k 


(k-1) 


|! (n-k) I  JO 


k-1,,  ,n-k, 

x  (l-x)  dx 


Mood  (i960),  page  235.  Letting  u 


t 

—  ,  we  have 
P 
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ph'(p) 


f 
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du  . 


SLnc 


1  -UD  h(p) 

so  £  1*  uicreaaing  In  p,  so  li  •  Thu«  if  A  k  out  of  n 

structure  t*  composed  of  independent  Uke  components  having  an  Increaeing 
failure  rate,  then  the  etrueture  itself  ha*  an  Increaeing  failure  rate. 


If  we  note  that  the  time  of  failure  of  a  k  out  of  n  eyatem  corres- 
pondi  to  the  kth  largest  in  a  sample  of  n  observations,  then  an  alternate 
statement  of  this  result  ie  the  following) 

Corollary.  Suppose  Xj4  X2<,  .  ,  <X  are  a  sample  of  order 
statiatice  baeed  on  Independent  obeervatlone  from  a  dlatrlbutlon  having 
Inerei.glna  failure  rate.  Then  the  dietributlon  of  hae  an  increaeing 
failure  rate.  1  ■  1,  2,  .  ,  ,  n.  — — —  —  — 

Actually  we  can  generate  new  etructuree  which  have  the  property 

that  1*  a  decreasing  function,  by  composition  of  structure*  having 

thi.s  property.  Under  composition  ws  form  a  supsr structure  each  element 
of  which  consists  of  copies  of  a  given  structure.  If  h  ■  £(g)  with  g'(p)^0. 
than  strtee 


hi  r>)  1(g) 


£ili£l 

ti(P) 


the  property  is  closed  under  compoettlon. 


Tho  following  example  show*  that  a  itructure  consisting  o',  inde¬ 
pendent  like  components  each  having  an  exponential  failure  distribution 
noed  Dpt  have  an  Increasing  failure  distribution, 

Example.  Consider  s  structure  composed  of  two  substructure*  in 
parallel,  the  first  having  k  components  m  series,  the  second  consisting 
of  a  single  component.  Assuming  independent  components  each  having 
exponential  dlatrlbutlon  for  failure 
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bB 


r(t)  ■  1  » •  *  , 

we  compute  the  probability  S(t)  of  etructure  vurvlval  past  time  t  to  be 


Thue 


S(t)  •  1  -  (1  -•"t)(l , 


-t 

• _ 


t  ka"kt  (h-H)e'(kt,1>t 
0-t  +  ,-kt  .  0*(k^Ht 


and  eo 

■gn  R'(t5  *  »gn  [-(k-l)Z  +  k2e-t  +  a"*'1  ] 


Note  that  for  k>l.  for  t  ■  0  the  aign  la  poaitive,  while  for  t  ■  oo 
the  aign  la  negative.  Thua  the  etructure  failure  rate  R(t)  la  not  monotonic 
for  k>l. 

Thla  ia  alao  a  counterexample  to  the  conjecture  that  k  out  of  n 
atructurei  with  unlike  IFR  component*  are  themaelvea  neceaaarily  IFR. 
(Simply  conaider  the  aerie*  eubatructure  aa  a  alngle  component. ) 

5.  STATISTICAL  TEST  FOR  MONOTONE  FAILURE  RATE. 
have  loan  the  impel  tant  rcu*.  that  diatrlbut.lona  with  me* civile  failure  rate 
play  In  reliability  theory.  Ttterefore  it  would  be  of  great  value  to  have  a 
teat  tu  determine  whether  t,  aamplu  (or  aet  of  eamplee)  cornea  from  a 
population  with  monotone  failure  rata.  Tha  taat  preaentad  In  thla  „.c»ion 
li  baaed  on  Proichan  and  Pyke. 

Let  .  X?,  .  .  . .  be  a  temple  of  Independent  obiervattona 

Lorn  the  common  Ulitrlbutlon  F  with  denaity  f,  whore  f(t)  *  0  lor  t<  0, 
and  failure  rate  r(t).  We  with  to  chooae  between  the  following: 

Null  Hypotheila,  Hq!  r  ia  conatant, 

Alternative  Hypothenii.  H(:  r  ia  non-decreaaing  but  not  conatant. 


i 


bo 
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The  test  statistic  is  computed  as  follows.  Lei  T^T  <C.  .  .  <CTn 

be  the  ordered  observations,  D  =  T, ,  D  =  T_ -T  ,  .  .  .  ,  D  =  -T 

1  1  2  2  ]  u  n  n-1 

the  spacings,  and  D  =  nD,  ,  D  =  (n-l)D„,  .  .  .  ,  D  =  D  .  the  normalized 

1  12  2  r>n 

spacings.  For  i,  j  =  1 ,  2 . n  let  V.  .  =  1  if  D.^>D.,  0 

_  .  .  .  11  31 

j. he  test  statistic  is 

n 

y  s  f  V 
n  A*.  .  i; 
i»J=l 

•-S; 

1  'J 


£• 
;i  iiV 


s 

I 
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Sk 

& 


We  reject  the  null  hypothesis  at  the  <X  level  c £  significance  if  V 

r  n  n,et. 

where  v  _  is  determined  such  that  P  V  ~>v  IH  =  £*.  (It  is 
n,«_  L  n  n,0C‘  o-> 

obvious  how  to  modify  the  Vn  test  if  the  alternative  hypothesis  in  that  r 
is  non-increasing  rather  than  non  -  dec  rea  sing. ) 

Heuristically  we  may  justify  the  test  as  follows.  Under  the  null 
hypothesis  it  may  be  readily  verified  that  Dj ,  D^,  .  .  .  ,  Dr  are  indepen¬ 
dent  exponential  random  variables  with  common  parameter,  say  X. 

Hence  P^V^.  =  1  j  =1/2  for  i,  j  =  1 .  2,  .  .  .  ,  n,  i  ?=  j.  However,  as 

shown  below,  under  the  alternative  hypothesis,  P  £v. .  =  l]]>l/2  for 

i^-j,  j  -  1>  2,  .  .  .  ,  n.  In  fact,  each  V..  and  V  tend  to  be  larger 

*>5  n  ° 

under  the  alternative  nypothesic,  so  that  rejection  of  the  null  hypothesis 
occurs  for  large  values  of  V  .  Since  under  the  null  hypothesis  the  dis¬ 
tribution  of  V  is  known,  we  have  available  v_  . 

n  n.oc. 

Distribution  under  the  Null  Hypothesis.  Since  under  Hr,,  D, ,  D  .  . 

_ ■  - - —  ~  1  2 

Dn  are  independently  distributed,  each  having  density  \e~  all 

orderings  of  D  ,  D  .  .  .  .  ,  15  are  equally  likely.  Using  this  prop*  r  ty. 

Kendall  (1938)  provides  tables  for  PjV  <  kjH  1  ,  n  <  10;  more  convenient 

t  n  —  0J  — 

2 

tables  are  available  in  Mann(1945).  Mann  shows  that  fl  and  ,  the  mean 

n  n 


and  variance  of  V  under  H  ,  are  civen  bv 
n  0 


Xj  -  -  Xn  F<Xt)  . 

IV  follows  that 

P[X|>u]-e*U. 

Thus  each  X|  Is  distributed  according  to  the  exponential  distribution 
with  unit  mean.  Moreover,  since  the  X  ,  ,  ,  ,  ,  X_  are  Independent,  sc 
are  the  X'.  ,  ■  .  ,  X1  ,  Next  lei  T'  <  T,<  ,  .  ,  <T'  represent  the  ranked 

X' ,  X!, . X'  so  ilia,  T'(  «  -  In  FfTj),  1-1,2  ,  .  .  ,  n,  Further, 

let 

TJj  ■  (n-i+lXT’-T^j),  1-1,2 . . 

where  ■  0  by  definition.  It  is  easy  to  vtuify  that  the  5^  are  in¬ 
dependently,  identically  distributed  according  to  the  exponential  with  unit 
mean,  See  for  example  Epstein  and  Sobel  (1953)  or  Rjnyi  (1953). 

Note  that  Ti  is  an  increasing  function  of  Moreover  TJ 

is  a  convex  function  of  as  shown  in  Theorem  4.  1,  Chapter  II  of 
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Barlow  and  Proschan  (1964).  It  follow*  that  Dj  7  fij  implies  5, 

for  i  <  j.  Thua  VU>V',  where  V'  «  1  if  5,  >  5,.  Hence  V„  y  V1  , 
ij  —  ij  i)  l  J  n  “  n 

where  V'  » £  V'  ,  »o  that  pfv  >v_  „  I  H,1 7  «l  for  04  of  4  1, 
n  ^  j  ij  L'  n  “  n,ot  '  1J“  — 

n  <  2i  3 . Implying  that  is  unbiased,  as  claimed, 


Asymptotic  Distribution  under  the  Alternative  Hypothasls.  Under 
the  alternative  hypothesis,  the  asymptotic  distribution  of  Vn  is  normal 

under  mild  restrictions,  It  follows  that  V  is  a  consistent  test, 

n 


The  asymptotic  relative  efficiency  of  the  V  tost  compared  with 

n 

various  possible  competing  tests  is  studied  in  Proschan  and  Pyke,  In 
general,  the  V ^  test  compares  quite  favorably. 
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UNBIASED  ESTIMATES  OF  RELIABILITY  WHEN  TESTING 


AT  ONLY  ONE  EXTREME  STRESS  LEVEL 


A.  Bulftnch 

Reliability  Engineering  Branch, 
Nuclear  Reliability  DlVieldn 
Quality  Aeiurance  Directorate 
Picatinny  Arsenal,  Dover,  New  Jersey 


ABSTRACT.  Baaed  on  the  atreas'itrength  concept  oi  reliability  tor 
"one  -  shot*1  items,  it  is  assumed  that  an  item  cannot  (ail  until  the  stress 
equals  or  exceeds  the  strength,  f  rom  this  premise  and  the  following 
additional  assumptions,  methods  are  given  tor  calculating  unbiased 
estimated  of  non-time  dependent  reliability! 

1.  The  relation  between  the  stress  and  strength  standard  deviations 
are  known  approximately. 

Z,  A  single  stress  level  is  applied  during  testing  at  approximately 
three  standard  deviations  from  the  average  stress  level. 

3.  The  stress  and  strength  distributions  are  normal. 

Calculations  are  included  to  show  the  effect  of  errors  in  the  assumptions 
concerning  the  standard  deviations,  applied  stress  level,  and  rounding- off 
errors. 

This  approach  further  reduces  the  sample  else  required  to  demonstrate 
nigh  non-time  dependent  reliability  in  laboratory  testing  It  has  the  ad-hid 
advantage  of  obtaining  unbiased  eetimatee  of  reliability  with  the  simplest 
oi  testing  methods. 

I,  INTRODUCTION,1  The  pressure  of  time  end  money  in  reliability 
testing  requires  s  never  ending  quest  for  simpler  methods  and  smaller 
sample  sites.  Recent  work  at  Picatinny  Arsenal  has  euggeeted  another 
contribution  to  this  effort. 

The  uoual  interpretation  of  sample  result*  for  the  determination  of 
non-time  dependent  reliability,  when  only  attribute  type  data  can  be 
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obtained,  ia  based  on  the  binomial  distribution.  The  usual  laboratory 
method  of  testing  is  to  apply  a  single  level  of  stress  to  the  sample. 

Under  these  conditions  very  large  sample  sices  ere  required  to  demonstrate 
reasonably  high  reliability  values  that  may  exist,  In  addition,  this  approach 
results  in  data  very  insensitive  to  changes  in  reliability,  Both  of  these 
characteristic*  are  costly  shortcomings,  However,  the  simple  method 
of  testing  is  an  asset, 

The  purpose  of  this  paper  is  to  describe  a  procedure  that  retains 
the  simple  testing  method  but  requires  only  small  sample  aiees  for  any 
reliability  level  and  produces  data  that  is  sensitive  to  small  changes  in 
reliability.  This  is  accomplished  by  changing  the  interpretation  of  the 
data  and  supplementing  this,  in  a  quantitative  way,  with  knowledge  grained 
from  tho  experience  of  working  with  an  item  Over  a  period  of  time. 

However,  the  method  presented  here  is  limited  to  the  laboratory 
detarmlnatlon  of  non-time  dependent  reliability  when  only  success- 
failure  typo  of  data  can  be  obteined.  This  type  of  reliability  le  baaod 
on  stress- strength  concept  presented  in  an  earlier  paper  (Reference  1). 

The  procedures  proposed  are  an  out-growth  of  recant  work  on 
ihv  evaluation  of  laboratory  method*  by  means  of  Monte  Carlo  sampling 
techniques,  This  work  showed  that  when  only  attribute  data  can  be 
obtained  that: 


1.  The  observed  proportion  of  successes  in  a  sample 
obtained  at  a  single  Htrcis  level  le  a  blast'd  estimata  of  the 
non-time  dependent  reliability  defined  by  tho  stress -strength 
concept. 


2,  A  sample  obtained  et  a  single  stress  level  cannot 
measure  the  average  or  standard  deviation  of  the  strength 
distribution. 

3,  The  observed  failure  rats,  obtained  at  a  single 
stress  level  measures  the  area  of  the  tail  of  the  strength 
curve  to  the  left  of  (below)  the  applied  stress  ordinate. 

From  the  above,  it  was  realised  that  eample  results  obtained  at 
a  single  stress  level  furnished  information  about  the  strength  distribu¬ 
tion.  This  suggested  the  possibllty  of  making  use  of  this  fact  for 
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obtaining  unbiased  cellmates  of  rallablllty,  with  the  very  simple  method 
of  teitlng  at  it  (ingle  (tress  level,  by  changing  the  ute  made  of  (ample 
results, 

II,  METHOD  OF  CALCULATION,  The  method  of  calculation  described 
below  is  baaed  on  the  normal  deviate: 


Where:  X^  a  Average  etreea  expected  in  use 

3?2  ■  Average  strength 

0 

(j  ■  Variance  of  the  stress  distribution 
a 

»2  ■  Variance  of  the  strength  distribution  . 

The  previous  work  referred  to  above  shows  that  precise  and  unbiased 
estimates  of  the  true  non>time  dependent  reliability  can  be  obtained 
by  entering  a  table  of  areas  under  the  standard  normal  curve  with  this 
calculated  T- value.  Tills  is  true  of  courae  only  when  the  stress  and 
etrongth  distributions  art,  normally  distributed,  Th-.  sensitivity  of  this 
(unction  to  deviationi  from  normality  is  yet  to  be  demonstrated. 

Since  testing  at  a  single  stress  Level  cannot  measure  the  *,*-r\«i* 
and  standard  deviation  of  the  strength  distribution,  the  above  formula 
wee  transposed  to  an  equivalent  function  as  follows: 

Let  X  ■  Any  applied  stress  lev*!  in 


testing,  Then: 


X 


7  fa 
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T  *  *  X  -  Xt  (tot  the  etreee  distribution) 


t  «  -  X  -X  <£or  the  strength  dintribution) 

*2  2  "  2 

X2  -  Xv  -  (X  -  Xt)  +  (X,  -  X) 

«  T  •,  +  T,»,  (by  eubetltution) , 

\  i  66 


When!  ms^  *  *2 

Y  .  JC  «  T.  *, 

x2  l  I  i 

T,»i  *  roVl 

T  ■  V*!  ♦ 

»,  It,  ♦  mTz) 

TL  + 

■  V'"  * » 

JHWMWIWj:  Th.  1...  »»»»'•  «*  »•  “•  toU°W‘nE 

assumptions: 

!  Tho  .tree,  end  strength  distribution,  ere  normal 
2,  Where  m.j  -  ■*.  m  is  known 

j.  The  testing  is  done  et  a  stress  o<  t*l  +  Vl*’  'Wh°r*  T1 
i„  known  approximately. 
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IV.  DISCUSSION  OF  ASSUMPTIONS’. 


1.  If  there  i*  reason  to  question  the  assumption  of  nigrmality, 
appropriate  distribution  free  methode  can  be  used.  However,  the  form 
of  the  distribution  should  be  determined  where  possible. 

2.  Experience  has  shown  that  m  is  approximately  two  The 
examples  given  below  show  that  the  value  of  m  can  vary  widely  before 
seriously  affecting  the  accuracy  of  the  resulting  reliability  value. 

3*  (X  +  T  •  )  can  be  defined  as  the  maximum  stress  expected 
in  use.  This  level  of  stress  is  usually  known  by  the  development  engineer 
or  is  specified  in  the  Military  Characteristics.  Such  a  maximum  stress 
can  be  defined  statistically  as  the  stress  occurring  only  once  in  a  thousand 
or  once  in  tan  thousand  times.  As  such,  T  a  3,09  or  T  *  3,72 
respectively.  The  examples  given  below  snow  that  T.  can  also  vary  widely 
before  seriously  affecting  the  accuracy  of  the  resultant  reliability  value, 

V.  USE  OF  MODIFIED  T  •  FORMULA.  In  the  above  formula.  T_ 

■  . .  . . - . -  / 

is  measured  by  the  observed  failure  rate  of  the  sample  tested  at  a  single 

stress  level  (X).  Its  numerical  value  can  b«  obtained  by  entering  a  table 

of  areas  under  the  etandard  normal  curve  with  the  proportion  of  failures 

in  ths  sample.  With  this  value  determined  and  tha  value*  of  T,  and  m 

known  or  assigned,  thr.  ab.ive  formula  can  be  used  without  knowing 

or  ig  the  average  and  standard  deviation  of  the  atrength  distietribution. 

The  average  and  standard  deviation  of  the  stress  distribution  must  \,j 
,'ieparately  determined.  If  this  information  is  not  available  and.  -anno*  be 
determined,  the  determination  of  a  numerical  value  for  reliability  is 
impossible. 

VI,  ACCURACY  AND  SENSITIVITY  The  incentive  for  using  the 
propoeed  method  of  calculating  reliability  is  that  it  can  furnish  consider¬ 
ably  more  Information  about  the  existing  reliability  than  the  usual  way  of 
using  sample  eucceis-failure  results,  Tlie  examples  given  In  Table  I 
show  thie  quite  well.  Obtaining  00%  sample  failure*  In  this  method  is 
not  as  bad  as  it  might  seem.  If  the  50ft  point  of  the  strength  curve  in 

at  the  three  (3.09)  sigma  point  of  the  stress  curve,  the  reliability  equals 
.9162  (when  m  equals  2)  -  not  50%,  the  proportion  of  successes*  in  the 
sample. 
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TABLE  1 

ACCURACY  ANP  SENSITIVITY 


Sample  Sice; 


n  =  22 


Number  of  Sample  Failure*!  3  b 

Standard  Deviation*!  2S^  a  S, 


Testing  Level;  Tj  ■  3,09(P  ■  .001) 


3,09  +  2Tg 


SAMPLE 

T  = 

T 

yi  +  <2)Z 

-FORMULA 

(1  -b/n) 

b 

b/n 

IL 

T_ 

Reliability 

Reliability 

11 

.  5000 

0,  00 

1.  38 

.9162 

,  5000 

10 

,4545 

0.  11 

1.48 

.9306 

.  5455 

9 

.4092 

0.23 

1.  59 

.9441 

.  5908 

8 

.  3636 

0.  35 

1. 69 

.  9545 

.6364 

7 

.  3182 

0.47 

1,  ?o 

.9641 

.  6818 

6 

.  2728 

C,  60 

1. 92 

.  6726 

.  7272 

5 

.  2272 

0.  75 

i .  U5 

.9798 

.  7728 

4 

.  1818 

0.91 

2. 19 

.9857 

.  8182 

3 

.1364 

1.10 

2,  36 

.9909 

.  8636 

2 

.  0909 

1.  34 

2.  58 

.9951 

.9091 

1 

.  0454 

1. 69 

2,  90 

,9981 

,  9546 

*  T-Formula  reliability  mlnu*  the  obeurved  proportion  of  ouccene* 


Dlffeyetice* 
+>4162 
+  3851 
+,  3533 
4.3181 
4.2823 
4.  2454 
*  2070 
+  .1675 
4.  1273 
4.  0860 
4.0435 

In  the 
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The  reeulti  in  Table  I  ehow  the  sensitivity  of  the  proposed  method 
to  changes  in  reliability  values.  A  decrease  of  .  03  in  the  reliability  at 
the  upper  end  of  the  scale  increases  the  number  of  failures  in  the  sample 
of  22,  from  cero  to  six,  This  is  a  significant  difference  at  the  95% 
confidence  level, 

The  above  sensitivity  is  to  be  compared  with  the  insensitivity  of  the 
method  of  using  the  observed  proportion  of  successes  in  the  sample  as 
the  point  estimate  of  11  reliability"  In  this  method,  where  the  binomial 
distribution  pertains,  the  success  probability  ( "reliability")  must 
decrease  approximately  0,23  (1,00  -  .77)  before  the  observed  number  of 
failures  in  the  sample  increases  a  significant  (0  to  5)  amount  at  the  95% 
level  of  confidence, 

The  above  comparison  of  sensitivity  shows  that  the  proposed  method 
is  sensitive  to  changes  in  reliability,  That  is,  the  proposed  method  can 
detect  relatively  email  changes  in  reliability  with  small  samplt  slaes, 

This  la  an  important  property  for  a  laboratory  method,  It  means  that 
email  differences  between  design  modifications  and  email  changes  occurr¬ 
ing  during  storage  can  ba  readily  detected. 

VII.  ERRORS  DUE  TO  ASSUMPTIONS,  The  relative  accuracies  of  the 
two  method  a  (or  determining  reliability  are  ehown  by  the  "differences" 
given  in  Table  I,  These  differences  are  to  be  compared  with  the  errors, 
resulting  from  incorrect  assumptions  shown  in  Tabic  II.  The  assumption 
errors  made  here  are  the  maximum  expected  in  practice  due  to  total 
*  Ignorance  about  the  system  concerned,  Any  knowledge  gained  about  a 
comporeet  ey  o  -';3tuiu  through  experience  wii)  in, provo  the  accuracy  of 
the  assumptions  and  thcirchv  reduce  the  resultant  errors.  This  kind  of 
knowledge,  from  experience,  is  always  available  and  can  be  t.'C'rtivuly 
ijxed  in  the  proposed  method  oi  calculation. 
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TABLE  II 

EFFECT  OF  ASSUMPTIONS  AS  FAILURE  RATE  INCREASES 

're it  Levels  U^  +  Tj 

Standard  Deviation:  mS^  ■ 


FAILURE 
RATE  (b/n) 

m 

II 

li. 

h 

T_ 

POINT 

ESTIMATE 

,  02 

1 

2.  33 

,0100 

2,05 

3.10 

,99903 

.  02 

1 

3.72 

,0001 

2.03 

4.08 

.99998 

.  02 

2 

3,09 

,0010 

2.03 

3.21 

,99934 

.  02 

3 

2,  33 

,0100 

2.03 

2,  68 

,99632 

.02 

3 

3,72 

,0001 

2.05 

3.  12 

,99910 

Mi.  iinum  Error; 

Assuming  the  molt  favorable  (higheet  reliability)  condition  when  In  fact 
the  irii'et  unfavorable  condition  actually  exieta;  .99998  «  .99632  *  +.00366, 

Median  Errorst 

Aecuming  the  median  (m  ■  2;  T,  “  3,  09)  condition  when  the  moat 
favorable  (1)  and  Unfavorable  (2)  conditions  exiat:  (l)  .99934  -  .99998 

=  - ,  00064  (?.)  ,  99934  -  .  99632  ■  +,  00302 
Effect  of  Assumptions  (continued)i 

Test  Levels  Uj  +  TjSj 


Standard  Deviation]  mS^  • 


4 


Design  of  Experiments  81 


FAILURE 
RATE  (b/n) 

.  05 

m 

II 

H 

h 

T_ 

POINT 

ESTIMATE 

1 

2.  33 

,  0100 

1. 65 

2.  31 

. 99752 

.  05 

1 

3,  72 

,  0001 

1.  65 

3,  80 

. 99993 

.05 

2 

3.  09 

.  0010 

1. 65 

2.  85 

. 99781 

.05 

3 

2.  33 

.  0100 

1. 65 

2,  30 

. 98927 

.05 

3 

3.  72 

.  0001 

1. 65 

2.  74 

.99693 

Maximum  Error; 

Assuming  the  most  favorable  (highest  reliability)  condition  when  in 
tact  the  moet  unfavorable  condition  actually  exists; 

.99993  -  .98921  =  +.01066 

Median  Errors; 


Assuming  the  median  {m  =2;  Tj  =  3.  09)  condition  when  the  most 
favorable  (1)  and  unfavorable  (2)  conditions  exist; 

(1)  .99781  -  .99993  =  ,00212 

(2)  .99781  .  ,98927  a  +.00854 

Effect  of  Assumptions  (ccr»*rmed); 

Test  Level;  Uj  +  TjE^ 


Standard  Deviation: 
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FAILURE 
RATE  (b/n) 

m 

II 

Ji 

T 

POINT 

ESTIMATE 

.10 

1 

2.  33 

,  0100 

1. 28 

2,  55 

.  99461 

.  10 

1 

3,  72 

.  0001 

1. 28 

3,  53 

. 99979 

.  10 

2 

3,  09 

.  0010 

1.28 

2.  52 

.99413 

.  10 

3 

2.  33 

.  0100 

1.28 

1.95 

, 97441 

.  10 

3 

3.72 

,0001 

1.28 

2,  39 

.  991 57 

Maximum  Error; 

A* turning  the  moit  favorable  (highett  reliability)  condition  when  in  fact 
the  molt  unfavorable  condition  actually  *xi*t*: 

,99979  -  ,97441  «  +.02538 

Median  Error* ! 

Aeiuming  the  median  (m  »  2;  Tj  ■  3.  09)  condition  when  the  mo*t 
favorable  (1)  and  unfavorable  (2)  condition*  exiatu 

(1)  .99413  -  .99979  «  -.00568 

(2)  .99413  -  .97441  *  +.01972 

Effn.it  of  Assumption*  (continued): 

Te*t  Level;  +  T^ 

Standard  Deviation:  mS^  =»  5^ 
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FAILURE 
[ATE  (b/n) 

m 

T1 

2jl 

_T_ 

POINT 

ESTIMATE 

.  20 

1 

2.  33 

.  0100 

0.  84 

2,  24 

, 98745 

.  20 

1 

3,  72 

.  0001 

0,  64 

3,22 

. 99936 

.20 

2 

3.  09 

.  0010 

0.  84 

2.13 

. 98341 

.20 

3 

2,  33 

.  0100 

0.  84 

1.  33 

, 93700 

,20 

3 

3.  72 

,0001 

0.  84 

1.97 

97938 

Maximum  Error! 

A*  Burning  the  moit  favorable  (hlgheet  reliability)  condition  when  in  fact 
the  meet  unfavorable  condition  actually  exietai 

,99936  -  .93700  ■  ,06236 

TABLE  III 

SUMMARY  OF  ERRORS 


h.rrore  Dui*  to  Uejng 

Sample  Proportion  of  Errora  Lue  to 

Succe»eee  ae  Point  AiBu.nptinne 


failure  Rate 

E  Btlmate 

Maximum 

Median 

,  02 

+  ,  019 

+  ,  004 

+.003 

.09 

+.  048 

+  .011 

: ,  008 

.10 

+  .094 

+  ,  025 

+  .  020 

.  20 

+.183 

+  .062 

+  .  046 
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Tho  ««mpla  errors  in  Table  111  were  obtained  by  subtracting  (1  -  b/n) 
from  the  point  estimates  (in  Table  11)  for  M  =  2  and  T^  *  3.  09  -  the  median 
conditions.  The  assumption  errors  in  Table  111  were  obtained  by  rounding 
off  the  corresponding  errors  in  Table  11. 

The  data  in  Table  111  show  that  both  types  of  errors  increase  as  the 
observed  proportion  of  failures  (failure  rate)  Increases.  However,  in 
each  case  the  assumption  errors  are  less  than  the  sampling  errors,  The 
magnitude  of  the  assumption  errors  up  through  a  failure  rate  of  0. 10  is 
not  g-eat  enough  to  seriously  affect  the  reliability  value,  Some  knowledge 
of  T^  or  m  will  greatly  reduce  these  errors  In  the  calculated  reliability. 

VIII.  EFFECT  OF  ROUNDING  OFF  ERRORS.  When  sample  siees  are 
email,  rounding  off  errors  may  be  important.  Their  effects  at  various 
failure  rates  are  shown  in  Table  IV  and  Table  V, 
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TABLE  IV 

EFFECT  OF  R0UND1NQ  OFF  ERRORS 
(SAMPLE  CALCULATIONS) 


T**t  Level!  +  3.  09  Sj 

Standard  Deviation:  2Sj  * 


_b_ 

n 

Js/il 

2i_ 

A 

T 

Point 

Estimate 

.45 

3.09 

,001 

+  .13 

1.  50 

.  9332 

5 

10 

.  60 

3.09 

.001 

.00 

1.  38 

.9162 

,  55 

3,09 

,001 

-.13 

1.27 

.8979 

_b_ 

n 

h/n 

A 

A. 

T„ 

*.  _ 

..L 

Point 

Estimate 

,  20 

3,  09 

,001 

,  84 

2. 13 

.  9:^4 

5 

?n 

^  *• 

.  V  J 

3.  09 

,  001 

6b 

1,99 

.  9767 

.  30 

3.  09 

.001 

.  53 

1. 85 

.  9678 

b 

n 

b/n 

Zi 

A. 

A 

JL 

Point 

Eatimnt-, 

.  05 

3.09 

,001 

1, 65 

2.  85 

,  OQ78 

9 

50 

.10 

3,  09 

.  001 

1.29 

2.  54 

.9945 

.15 

3.  09 

.  001 

1.04 

2.  31 

,  9896 
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Median  Error*; 

Assuming  the  median  (m  *  2|  ■  3,  09)  condition  when  the  moat 

favorable  (1)  and  unfavorable  (2)  condition*  exist: 

(1)  ,98341  -  ,99936  ■  *.01595 

(2)  .98341  -  .93700  ■  +.04641 


TABLE  V 

SUMMARY  OF  EFFECT  OF  ROUNDING  OFF  ERRORS 
Test  Level:  Uj  +  3.09Sj 

Standard  Deviation:  2S^  = 


FAILURE  RATE 

.05 
,  10 
,  20 
.  30 
.40 
.  00 


MAXIMUM  ERROR 

.  0006 
.0082 
.  0128 
.0185 
.0250 
.0353 


The  errors  shown  in  Table  V  are  the  differences  between  the  max¬ 
imum  and  minimum  reliability  values  for  each  fallur*  rate  (b/n).  The 
method  of  calculating  the  maximum  and  minimum  values  is  baaed,  on  th« 
assumption  of  rounding  off  error*  of  +  0.  05  in  the  failure  rate  as  shown 
in  Table  IV. 
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Although  the  eesumed  rounding  off  error  is  the  maximum  expected, 
its  magnitude  is  not  excessive  below  a  failure  rate  of  0,  30,  As  shown  in 
Table  V,  this  type  of  error  also  increases  with  the  failure  rate, 

IX.  USE  OF  CHEBYSHEV'Si  INEQUALITY,  There  is  little  nr  no 
Information  available  on  the  form  of  strength  distributions  of  most  missiles 
and  missile  components,  Furthermore,  it  is  costly  to  obtain.  It  would  be 
helpful  if  a  distribution  free  procedure  such  as  Chebyshev's  irrequelity 
could  be  used,  As  shown  in  Tabls  VI,  the  use  of  Chebyshev's  inequality 
in  the  modified  T- formula  resulted  in  ridiculous  values. 


TABLE  VI 

CHEBYSHEV'S  INEQUALITY 


Test  Levoli  Uj  4  TjSj 
Standard  Deviation;  2Sj  ■ 


0.S0  31.62  .001  2  IS. 4 

The  T  value  of  IS,  4  r.hown  in  Tible  VI  is  to  be  compared  with  the 
T-vslue  of  1.  38  shown  in  Tabls  1  tor  a  failure  rate  of  0.  SO.  From  this, 

:t  is  concluded  that  Chsvyshev's  inequality  cannot  be  used  In  thle 
application. 

X,  EXAMPLES.  Previous  work  (Reference  l)  hes  ehown  that  the  true 
non-time  dependent  reliability  of  the  aet  of  conditions  used  in  these 
example*  can  be  obtained  by  means  of  the  following  formula; 

U,  -  u, 


4 
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Where: 


Ut  »  True  mean  of  the  (tree*  distribution 
2 

5^  a  True  variance  of  the  etreat  distribution 

U2  =  True  mean  of  the  itrength  distribution 
2 

S,,  b  True  variance  oi  the  strength  distribution, 

The  reliability  value  obtained  by  means  of  the  above  formula  was  used 
to  determine  the  accuracy  of  the  following  two  methods  of  using  attribute 
data  obtained  from  the  application  of  a  single  stress  level: 

1.  Using  the  observed  proportion  of  successes  in  the  sample  as 
the  reliability  point  estimate 

2,  Using  the  observed  proportion  of  failures  in  the  sample  as 

a  measure  of  the  area  of  the  strength  distribution,  bulow  the  applied  stress, 
to  obtain  T^  in  the  T-formula, 

The  errors  associated  with  the  two  methods  of  using  sample  data  are  to  he 
compared  to  show  the  practice)  value  of  the  method  proposed  here. 

The  condition*  ueed  in  1H«  example  are: 

3tr*»*  Strength 

Uj  *  10  U?  »  42 

S.  “  s  »  ■  to 

1  2 

The  true  non-time  dependent  reliability  for  thi*  set  of  condition*  can  be 
calculated  ae  follow*: 


i 


* 


Deiign  of  Experiment! 


8  <r 


Z 


42  -10 


*  V&O)2  +  <5)2 


32 
1 25 


2.  86  , 


The  true  reliability  associated  With  thii  Z-value  ii  0,  9979. 

1,  First  Method 

Uiing  the  observed  proportion  of  eucceeeee  si  the  point  estimate: 

If  it  is  assumed  that  the  testing  is  done  at  4  3Sj,  then 

the  applied  stress  will  be  equal  to  28  units.  For  the  set  of  conditions  des¬ 
cribed  above,  the  portion  of  the  strength  distribution  below  28  unite  can  be 
found  a  a  follows: 

Za  «  ^  *  SJL  -  1.70. 

Entering  a  table  of  areas  under  the  standard  normal  curve  with  this  Z 2 
value,  the  following  value  is  obtained! 

P  *•-  .  0446 


The  earlier  work  referred  to  above  shows  that  this  latter  vr.lun  is  the 
expected  failure  rate  of  the  single-etrese-level  method.  The  complement 
«;f  this  veluo  (,  9584)  would  he  taken  aa  the  "true"  mean  rellsbiiity  of  this 
method.  The  difference  between  0.  9979  and  0  9554  (U.  0423)  le  considered 
the  expected  error  of  the  *j"£le-atrr  m-level  method  when  the  proportion 
of  succeieas  in  the  sample  is  taken  *  the  point  estimate. 

2.  .Second  Method 

Using  the  observed  proportion  of  auccessea  as  a  measure  of  the 
uvea  in  the  tail  of  the  strength  curve: 

The  practical  value  of  the  method  proposed  here  can  best 
be  demonstrated  by  calculating  the  magnitude  of  the  errors  due  to  the 
eaeumptions  made  concerning  m  and  Z^.  Using  the  set  of  conditions 

described  above,  the  variations  in  m  and  Z^  utod  below  ore  the  max- 
mum  considered  likely  in  practice.  Therefore,  the  errors'  in  the  reliabil¬ 
ity  values  caused  by  theee  variations  are  the  maximum  expected. 
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TABLE  Vll 

VARIATIONS  DUE  TO  ASSUMPTION  ERRORS 


Failure 

Failure  Rate 

I2. 

m 

Z1 

A 

T 

Point 

Estimate 

1/22 

1,70 

1 

2.  33 

.0100 

2.85 

.9978 

l/22 

1,70 

1 

3.  72 

.0001 

3.  83 

.  9999 

1/22 

1, 70 

3 

2.  33 

,0100 

2.  35 

.9906 

1/22 

1.70 

3 

3.72 

,0001 

2,79 

.9974 

The  following  error*  were  obtained  by  calculating  the  difference*  botween 
the  true  value  and  the  point  estimate*  ahown  in  Table  VII: 

ERRORS 

M  Z1  Difference* 


1 

2.  33 

.0001 

1 

3,72 

.0020 

3 

2.  33 

.007  3 

3 

3.72 

.  OuuZ 

These  error*  are  to  be  compared  with  0,  0425,  the  error  obtained  when 
the  sample  result  was  used  as  the  point  estimate  of  nun-time  dependent 
reliability. 
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XI.  CONCLUSIONS. 

1.  The  proposed  use  of  attribute  data  to  estimate  non-time  dependent 
reliability  by  the  single-stress-level  method  is  more  accurate  at  all  levels  of 
reliability  then  the  usual  method  of  using  the  proportion  of  successes  in  the 
sample  as  the  reliability  point  estimate. 

2.  The  proposed  method  is  more  sensitive  to  changes  in  reliability 
than  the  usual  method. 

3.  The  proposed  method  permits  the  knowledge  gainod  through  the 
experience  of  working  with  an  Item  to  be  used  in  a  quantitative  way  and. 
thereby  reduce  the  sample  else  required  to  obtain  an  unbiased  estimate  of 
reliability. 


4,  When  the  true  reliability  of  an  item  is  in  fact  as  high  as  0,  995 
(the  usual  value  of  Military  Characteristics  requirements)  and  the  stress 
is  applied  at  the  three-sigma  level,  the  expected  error  in  the  proposed 
method  is  less  than  1.  0%, 
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COMMENTS  ON  THE  STRENGTH  OF  REAL 
STRUCTURAL  MATERIALS 


Kenneth  H.  Abbott 
Army  Materials  Research  Agency 
Watertown  Arsenal ,  Watertown,  New  York 


[Remarkr:!  Following  the  talk  by  A.  Bulfinch  there  wan  a  lively  discussion, 
In  the  light  of  the  questions  poeert  the  following  comment*  ihould  be  of  interest 
to  thoee  attending  Session  I, 

The  most  common  structural  materials  for  military  equipment  are 
metals,  As  a  result,  the  following  comments  are  specifically  related  to 
structural  metals  and  alloys,  although  they  are  applicable  in  a  generic  sense 
to  other  structural  materials. 

The  strength  of  an  engineering  alloy,  or  more  generally,  the  strength 
distribution  for  a  specific  alloy  and  processing  treatment,  is  normally 
determined  by  tensile  testing.  During  a  tensile  test  to  determine  the 
maximum  stress  that  can  be  lulsratsd  before  failure  occurs,  the  general 
behavior  of  the  alloy  is  illustrated  schematically  in  Figure  1,  a  typical 
stress-strain  curve.  The  engineering  yield  strength  Is  defined  as  the 
{•trees  (load  per  initial  area)  after  a  predetermined  amount  of  plastic  flow 
lies  taken  place.  The  ultimate  tensile  strength  is  defined  as  the  stress  at 
thv  maximum  load,  and  normally  occurs  after  an  additional  amount  of 
pie *tic  flow,  After  reaching  the  ultimate  tensile  strength,  fhu  applied  load 
generally  decrease;  as  local  necking  develops,  additional  plastic  flow  occurs, 
and  finally  the  specimen  fract"*,r*.  Strength,  then,  is  normally  determined 
by  the  plastic  propertiss  of  the  metal  and,  per  ne,  is  not  relateo  to  fracture. 

Ideally,  structures  should  behave  in  a  similar  fashion,  i.  r.  ,  U  a 
sufficient  load-carrying  cross  section  is  available,  the  structure  should 
yield  (undergo  plastic  flow)  before  it  fracture!,  This  simple  approach, 
however,  is  complicated  by  the  geometry  of  the  structure  and  assc,-Uted 
stress  field,  ao  well  as  by  fabrication  and  processing  defects  such  as 
small  voids,  inclusions,  weld  cracks,  etc.  In  an  actual  structure,  the 
propagation  of  a  crack  from  an  existing  defect  can  result  in  fracture  of 
the  metal  before  any  appreciable  general  plaetlc  flow  takes  place,  As 
Indicated  schematically  in  Figure  2,  fracture  can  occur  at  a  stress  level 
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below  the  engineering  yield  strength,  This  phenomenon  of  the  occurrence 
of  fracture  without  any  general  plastic.  flow  i«  defined  as  "brittle  fracture  " 

Hence,  the  assumption  that  component  failure  always  occurs  when  a 
particular  strength  in  exceeded  (normally  the  yield  strength)  Is  invalid. 
Numerous  examples  are  reported  in  technical  literature  of  both  military 
and  civilian  structures  that  failed  by  fracture  at  nominal  stress  levels  well 
below  the  yield  strength.  Some  Instances  are  reported  where  such  fracture 
failures  or  urrod  at  stress  levels  of  only  20%  of  the  yield  strength, 

Minute  defects  from  which  brittle  fractures  originate  cannot  be  elimin¬ 
ated  by  non-destructive  tasting  techniques  for  two  reasons.  The  first  is 
that  some  such  defects  are  below  the  resolution  limit  of  the  non-destructive 
to  sting  equipment.  The  second  is  that  many  defects  can  originate  subsequent 
to  component  manufacture  by  various  time  dependent  processes  such  as  cor¬ 
rosion,  hydrogen  embrittlement,  etc.  Hence,  the  design  of  structures  which 
fail  through  plastic  deformation  rather  than  by  fracture  is  a  very  difficult 
and  complex  process. 

The  following  references  arc  suggested  for  additional  det«M' 

1.  American  Society  for  Testing  Materials,  "Symposium  on  Effect  of 
Temperature  on  the  Brittle  Behavior  of  Metals,"  ASTM  No.  158,  June  1953. 

2.  "Conference  on  Brittle  Fracture  In  Steel,  "  West  of  Scotland  Iron 
and  Si '•el  Institute,  May  1953. 

3.  "Symposium  on  Prevent!.-.,  of  Brittle  and  Lew  Temperature  Failures 
of  Steel  Components,"  Watertown  Arsenal  Laboratories,  May  1949, 

4.  Cottrell,  A.  VI.  ,  "Theory  of  Brittle  Fracture  in  Steel  and  Similar 
Metals."  Transactions  AIME,  V  212,  195B. 

5.  "Fracture  of  Structural  Metals,"  WAL  MS-49,  June  1962. 
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THE  COLLECTION,  PROCESSING  AND  STORAGE  OF  DATA 
ON  SERIALIZED  NW  ITEMS 

Manfred  W.  Krimmer* 

Statistics  and  Analysis  Section,  Reliability  Branch,  OAD 
Pieatinny  Arsenal,  Dover,  New  Jersey 

The  existence  of  *  requirement  for  a  standardised,  usable  means  of 
data  collection  and  processing  for  Serialised  Nuclear  Weapons  Items 
necessitated  the  writing  of  a  Quality  Assurance  Instruction  (QA1-PA-1) 
and  two  major  computer  programs,  namely,  an  Error  Program  and  a 
Summary  Program.  This  paper  will  outline  the  application  of  QAI-PA-1 
to  Serialised  NW  Items,  the  fundamentals  of  the  Error  and  Summary 
Computer  Programs,  and  the  results  obtained  therefrom. 

1.  The  Quality  Assurance  Instructioh  standardises  the  method  of  data 
collection  for  those  Serialised  nuclear  munitions  produced  under  die  Juris¬ 
diction  of  the  U.  S,  Army  Munitions  Command.  The  automatic  data  pro¬ 
cessing  system  Includes  the  standardisation  of  forms,  recording  methods, 
and  conversion  of  data  to  punch  cards  for  transmission  to  Picatinny 
Arsenal.  Computation,  analysis  and  interpretation  of  the  data  for  quality 
and  reliability  evaluation  then  taken s  place.  The  information  cblleetad 

for  each  item  includes  acceptance  data  and  performance  results  a*  well  as 
stockpile  test  information. 

2.  The  QAI  has  three  primary  purposes, 

a.  The  Reporting  of  Identification  Information  and  Test  Result-. 

h.  The  reporting  of  Drawing  Change  Documents  and  Associated 
Serial  Numbe"- 

c.  The  Reporting  of  Related  Serial  Numbers. 

Poliowing  is  s  brief  explanation  of  the  above  reports  covering  The 
Reporting  of  Identification  Information  and  Test  Results  last,  it  being  the 
primary  report  of  this  system,  and  *  discussion  of  the  computer  •  vogrsms 
and  what  is  done  with  the  results. 

3.  The  reporting  of  Drawing  Change  Documents  such  as  englnserlng 
orders  and  technical  data  change  requests  and  the  serialised  iteme  pro- 


*At  the  present  time  the  author  ie  with  the  Ammunition  Procurement 

and  Supply  Agency,  Joliet,  Illinois. 
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duccd  under  each,  was  requested  by  the  reliability  engineers  at  Picatinny 
Arsenal  to  aid  them  in  more  accurately  evaluating  the  stockpile  reliability. 

This  is  done  on  reporting  forms  SMU PA  NS-90Q  and  NS-900A.  The  first 
form  (Fig.  1)  SMU  PA  N3-900  is  a  record  of  all  Drawing  Numbers,  re* 
visions  and/or  EO's  or  TDCR  numbers.  At  the  start  of  production  or  pro* 
ductlon  under  a  new  contract,  the  contractor  will  list  all  the  appropriate 
documents  under  which  proauction  will  begin.  The  second  form  (Fig.  2) 

SMUPA  NS-900A  lists  all  the  serial  numbera  of  the  components  produced 
under  the  drawing,  revisions  and/or  EO's  or  TDCR'a  recorded  on  the  first 
form  . 

Upon  receipt  of  one  or  more  EO's  (or  TDCR'a),  another  primary  card 
form,  SMUPA  Form  900,  is  filled  out  listing  these  additional  EC's  (or 
T .OCR's).  A  listing  of  the  component  serial  numbers  for  units  produced 
under  the  new  EO'a  (or  TDCR'e)  is  then  recorded  on  a  new  secondary  card, 
SMUPA  Form  900A.  When  additional  EO's  (or  TDOR's)  are  received, 
another  primary  card  form  is  complstsd  listing  only  thsss  additions,  Similarly 
a  listing  of  the  componsnt  ssrlal  number,  for  units  produead  under  these  EO's 
(or  TDCR'e)  is  recorded  on  a  new  secondary  card. 

This  procedure  le  continued  until  three  months  have  elapsed.  At  a  logi¬ 
ns  1  point  In  production  (i.  s. ,  prior  to  incorporation  of  a  new  EO  (or  TDCR)) 
all  tne  preceding  primary  cards  and  their  associated  secondary  cards  are 
forwarded  to  Picstinny  Areenal  in  punch  card  format.  The  above  pro¬ 
cedure  is  then  repeated,  beginning  with  an  initial  primary  card  listing  all 
the  (’.’-awing  numbers  with  their  up-to-date  revisions,  and  Engineering 
Ordei  Numbers  (or  Technical  Data  Change  Requests)  not  yet  associated  will, 
a  drawing  change .  under  wh!c.i  production  will  be  continued.  This  process 
is  repeated  continuously  every  three  mouths  or  until  the  end  of  production. 

The  information  is  then  put  on  punched  cards  and  submitted  by  the  producer. 

•?.  The  Reporting  of  Related  Serial  Numbers  ie  necessary  to  main¬ 
tain  an  accurate  up-to-date  record  of  what  assembly  an  item  is  a  part  of 
throughout  its  life  cycle  and  what  items  make  up  a  specific  assembly, 

This  information  is  required  by  MUCOM  through  direution  of  the  Army 
Materiel  Command.  The  information  is  reported  on  SMUPA  N9-901 
(Fig,  3)  and  lists  the  aerial  numbers  of  all  the  components  making  up  a 
specific  assembly.  This  information  is  submitted  by  the  contractor  assem¬ 
bling  the  item  and  by  field  personnel  whenever  s  component  le  removed  or 
replaced. 
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5.  The  last  report)  but  most  Important)  in  the  ayitem)  ta  the  reporting 
of  Identification  Information  and  Teat  Results,  This  information  la  reported 
on  SMUPA  NS -902  <Fip-  4).  This  la  the  basic  douument  for  recording  the 
results  of  all  the  Inspections  for  which  Information  is  required.  It  contains 
all  information  necessary  to  identify  the  item  such  as.  contractor,  contract 
number,  spec,  number  and  date  of  inspection.  It  also  contains  the  variables 
test  results  at  called  for  in  the  purchase  description,  it  identifies  the  de¬ 
lect,  and  gives  the  disposition  of  the  item.  This  Information  is  recorded  by 
the  item  contractor  and  submitted  on  punched  cards. 

6.  To  simplify  the  identification  of  the  tests  called  for  in  the  purchase 
description,  a  six  digit  "test  identification  code11  was  developed  through 
numerous  conferences  with  engineers  concerned.  This  coda  la  contained 
in  the  PD  and  ie  made  up  as  follows.  Sines  all  the  tests  in  the  PD  are 
called  out  in  paragraph  3  and  are  numbered,  for  example  3. 10, 1.  3,  the 
first  digit  signifying  paragraph  3  is  omitted  and  the  code  begins  with  the 
next  two  digits  signifying  the  first  subparagraph.  The  next  eubperagraph 
is  then  designated  by  a  single  digit  and  the  last  thr*e  digit*  of  the  coda  are 
individual  tests  in  numaricsl  sequence,  For  example,  10/1/002  identifies 
the  above  paragraph  and  the  second  test  within  that  paragraph.  The  codes 
are  unique  within  PD's  but  not  among  the  various  PD'e, 

7.  During  the  gathering  of  this  data  by  the  item  coruractor,  monthly 
shipments  of  punched  cards  containing  Identification  Information  and  Test 
Results  corns  into  our  office.  These  cards  must  be  checked  for  various 
errors  in  recording  and/or  key  punching.  This  is  accomplished  by  means 
oi  nn  Error  Program.  The  information  on  the  cards  is  put  on  magnetic 
tape  and  fed  into  the  computer  and  the  Error  Program  checks  each  card, 
column  by  column,  for  information  which  does  not  belong.  If  all  the  cards 
cue  correct,  analyaation  of  the  data  takes  place.  If  there  are  r-rar#  in  the 
curds,  the  computer  will  print  out  what  the  errora  are  and  where  to  finu 

The  cards  are  then  corrected  by  ue  if  the  volume  of  errors  is  not. 
too  great  or  they  are  sent  back  to  the  contractor  for  correction. 

8.  If  the  date  ie  good,  it  is  analysed,  This  ie  done  using  our  Nummary 
Program  which  takes  the  data  and  computes  the  following! 

s.  Lists  all  readings  which  are  out  of  specification  limit*, 

b.  Computes; 

(1)  Mean 
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(2)  Standard  Deviation 

(3)  (i)  lo-  ,  2  O'  and  3  O'  limit*. 

c.  Print*  out  the  max.  and  min.  value*  and  the  rang*. 

d.  Lilt*  all  reading*  outside  of  the  sigma  limit*. 

* 

e.  Print*  out  a  frequency  distribution  based  on  ?  sigma  limits. 

f.  Indicate*  skewness,  which  tests  ths  normality  of  the  curve. 

g.  and  indicates  kurtosi*  which  measure*  lire  flatnoss  of  the  curve. 

This  summary  program  is  run  for  each  lot  of  data  rscsivsd  on  a  monthly. 

basis. 


9.  A  second  type  of  Search  and  Summary  Program.,  providing  the  earn* 
output  as  th*  one  above  is  run  for  the  entire  production  or  for  any  special 
casus  where  specific  information,  is  required. 

10.  Ths  results  obtained  are  then  studied  and  interpreted  and  a  report 
i?  written.  The  reports  are  issued  and  a  follow  up  action  is  maintained 
until  replies  to  the  reports  recommendations  are  received  indicsting  some 
action  being  taken. 

Th*  data  is  then  retained  o..  matter  tapes  and  stored  fnr  future  use. 

It  is  felt  that  this  system  i*  a  goad  one  and  it  l*  Intended  to  increase 
it*  capabilities  end  scope  as  future  development*  erle*. 

Ha  quests  tor  copies  uf  QAI-PA-1  or  any  further  information  about  this 
system  ehould  be  directed  tot 


Commanding  Officer 
Picatinny  Arsenal 
ATTN:  SMUPA-NR2 
Dover,  New  Jersey 
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PROBABILITY  THAT  STRESS  IS  LESS  THAN  STRENGTH  AT 
PRESCRIBED  CONFIDENCE  LEVELS,  FOR  NORMALLY 
DISTRIBUTED  DATA 

E,  L.  Bomba  ra 

Engine  Projects  Office,  Marshall  Space  Flight  Center 

SUMMARY,  Under  cognieance  of  Marshall  Space  Flight  Center,  the 
stress  vs,  strength  concept  is  often  employed  in  small-sample  testing 
during  design  of  engine  subsystems. 

Under  the  assumption  of  normally  diewiouicu  data  taken  from  small 
samples  of  expensive  items,  this  paper  present*  method*  for  estimating 
probability  that  stress  does  not  exceed  strength,  with  a  prescribed 
level  of  confidence,  Tables  are  provided  to  simplify  calculations  of 
such  probabilities i 

Input  data  are  the  mean  and  standard  deviation  of  operating  streas 
from  na(  tests,  and  the  mean  and  standard  deviation  of  strength  from 
n^  tests,  Precision  of  these  methods,  based  on  approximate  results 
by  Welch  and  Aspln,  is  indicated  vs,  sample  sizes  and  ratio*  of  vari¬ 
ances,  By  examination  of  Tables  1  and  11,  sample  elees  required  for 
stress  and  strength  teats  can  be  determined,  As  one  might  expect,  if 
a'WO'se^'  it  best  to  select  n,^ 4  nBg  in  order  to  achieve  highest 
precision,  and  conversely  for  O' 2 , a/ cf2^1  Figure  III),  Compari¬ 
son  of  analytical  results  with  computer  simulations  (Table  I)  have  been 
mode  to  illustrate  the  amount  of  bias  of  the  method  when  the  number  of 
.cats  Is  very  small;  say,  5, 
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INTRODUCTION.  Under  the  concept  promoted  by  Robert  Luaser  of 
the  Army  Ordnance  Micelle  Command  (reference  1),  a  meaeure  of  aub- 
syetem  reliability  can  be  eatimalad  through  meaeurement  of  aafety  margin* 
when  data  i*  normally  dietributed.  For  example,  euppoee  that  in  develop¬ 
ment  teet*  of  a  apace  vehicle  the  yield  point  (etrength)  of  aeveral  presaure 
veeaele  of  the  aame  deeign  (e.  g.  ,  fuel  tanka,  thruat  chambera,  etc. )  la 
meaaured  and  that  the  operating  environmental  lavela  of  theae  (atreaa) 
are  meaaured  in  actual  flight*  or  in  etatic  teet*.  Certainly  it  would  be 
comforting  to  know  that  all  meaeurement*  of  the  latter  type  are  lea*  than 
all  of  the  yield  point*,  ae  ahown  in  figure  I.  The  bell-ahapad  cutva  indi¬ 
cate*  the  aaeumption  that  atreaa  and  atrength  value*  are  normally 
diatributed. 


*  Yield  Level 
(Strength) 

Figure  I 

Meaaurementi  of  Strea*  and  Strength 

We  might  be  contented  with  thi*  situation,  or,  dependent  on  the  veh'cl- 
miaaion,  w*  might  atlll  fear  the  not -too -remote  chance  that  atreaa  exceed* 
atrength  on  the  next  equipment  to  be  tested.  Certainly,  if  we  observe  over¬ 
lap  of  the  atruaaes  and  atrength*  we  ahould  be  concerned,  On  the  other 
hand,  if  all  strength*  exceed  all  atreaa**  by  a  large  amount,  the  deeign 
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might  be  relaxed  (reference  2).  We  are  therefore  intnreated  in  the  prob¬ 
ability  that  atrei*  it  leta  than  strength  in  a  tingle  mission.  We  should  also 
be  concerned  with  confidence  level,  due  to  the  necessarily  small  sample 
sixes. 

Let  us  define  the  observed  scatter  of  stresses  and  strengths  in  terms 
of  their  respective  sample  standard  deviations,  t  and  s^(  ,  with  corres¬ 
ponding  observed  means,  Xlt  and  5Fth.  The  probability  that  stress  is  lost 
than  strength  may  be  demonstrated  at  a  given  confidence  level  under  the 
assumption  of  normally  distributed  atrasssa  and  strengths,  as  a  function  of 
*th‘  *»*'  •  th'  "ss  '  and  respective  sample  sixes,  nth  end  ngg  ,  For 

brevity’s  take,  define  probability  that  stress  is  less  than  strength  an 
"stress-strength  reliability."  As  indicated  in  th*  appendix,  stress- 
strength  reliability,  ft,  can  bu  demonstrated  (or  not)  at  a  prescribed  con¬ 
fidence  level,  C,  in  terms  of  n^,  ngi  ,  and  the  stntistlc 


ID 


ft 


‘th 


'll 


v=s 


.h  * 


relative  to  K>  If  K  *  K,  R  is  demonstrated  at  confidence  level  C, 

This  paper  deals  primarily  with  methods  of  finding  K  as  a  (unction 
oi.it,  C,  ssmple  sixes,  and  variances.  0-Jj  is  ths  population  variance 
of  strength,  intimated  uy  s“n  from  n^  observational  Cggz  is  the 
population  variance  of  stresr  estimated  by  eggz  from  n>g  observations, 

ILLUSTRATIVE  EX AMPLE  S, 

Case  It  nth  »  ngf  «  n  |  b«  /  (XggZ  Unknown, 


If  the  number  of  stress  and  strength  measurements  are  the  same,  the  pro¬ 
cedure  it  relatively  simple: 

It  it  desired  that  R  ■  99  be  demonstrated  at  90%  confidence.  Data  Is 
as  follows: 


V 
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7.  22 

*th  ■ 

13, 

28 

•  1  ~ 

1. 040 

#th  " 

3. 

120 

at 

III 

“th  1  "  ' 

7, 

From  (1),  ft  ■ 


13,28  •  7,22 
V  3  120^  +  1. 040z 


1, 843  „ 


Next  calculate 

Effective  Sample  Size 

(2) 


f  +  1 


2  >2 


(n-l)(»th+  »“B) 


,4  +  .4 

‘th  +  * 


+  1 


«  I 


fe]3.1202  +  1.04Q*)2  +  j  ,8,32. 

3.  1204  +  1. 0404 


Enter  table  III  ualng  confidence  ■  90%,  R  ■  .  99,  and  conservatively , 
effective  sample  alee  otd  to  find  K  ■  3.78. 

Since  ft  <  K,  .  99  street -strength  reliability  Is  not  demonstrated 
at  the  90%  confidence  level,  heading  to  the  right  in  table  III,  we  see 
that  for  K  ■  1.83.  R  ■  .  84,  in  this  case  ft>K,  and  ,  84  stress- 
sivtmgth  reliability  is  demonstrated  at  90%  confidence.  The  50%  cot, 
fidence  K-vslue  for  effective  sample  sire  8  in  table  UI  of  1. 83  gives 
R  at  50%  confidence  ■  .  96, 

2  2 

Case  2i  n  titles*  ^  *  cT£h  /  0"it  Unknown, 

Tf  the  number  of  etress  njeaeuremente,  ng#,  and  strength  measure¬ 
ments,  n(h,  are  not  equal,  K  is  deftned  as  in  case  1,  but  the  procedure 
for  finding  K  is  more  complex: 
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Using  the  data  of  figure  I, 


From 


*s» 

«  7. 

22 

X 

th 

"as 

a  l. 

040 

*th 

nss 

a  9 

nth 

ft,, 

13. 

smmmss 

7.  22 

1202  + 

1. 040 

2 

13.  28 
3.  120 
5 

1.  843, 


(Again,  suppose  we  wish  to  determine  whether  R  «  .  99  @  C  "  .  90). 
From  a  table  of  the  cumulative  normal  distribution  (e.  g.  ,  reference  9) 
find  the  one-sided  value  of  99  *  2.  32t 


Calculate  the  parameter  of  non -centrality  of  the  non-central  t  statistic! 

(3) 


5.  320. 


Calculate  effective 

degrees 

of  freedom! 

<nth*th 

+  ns«*»«2' 

f 

„2  «4 
nth*th 

ft2  *** 

"th"1 

".s'1 

5.  647 

# 
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Calculate 


6,  320/^11,29 

\/nu° 

V  11,29 


-  0. 8452  . 


Find  A  in  table  IV  for  £  ■  1  «  C  *  .  10  ae  a  function  of  *1  and  f.  Interpolate 
linearly  on  ,  then  interpolate  on  lll'fi,  unless  f  < 9 ,  in  which  case  linear 
interpolation  on  f  is  sufficient  (see  reference  7).  In  this  example,  tor  t  ■ 
5.643,  two-way  linear  interpolation  is  sufficient,  giving  A  *  1,362. 


Then 

(6) 


$  4  A.VT7&/»r> 

(1  -  A2/2f)  $  /ZR 


Notr  livtt  $ /ZR  is  the  radical  in  equation  (3). 

5.320  +  1.  362Vl  +  5,  3201Z/1  1. 29  -  1,  362Z/ll,  2?  .  ,, 

K  « - ,  . . — - 1 - -  «•  iV 

(1  -  1.  36E2/  1 1 .  29)  3.  320/Z.  326 


Since  $  <  K,  .99  streee-strength  reliability  is  not  demonstrated  »t  90% 
confidence  level.  By  trial  end  error  we  may  reduce  ZR  enough  that  R  i» 

demonstrated  at  90%  confidence;  0.  g  let  ZR  *  .  8779.  corresponding  tc  R  = 
0.81  £  a  5.  320  (.  87791/2.  326  -  008.  Then 

fl  «  (2.  008  /Vll,  29)/VM'(2.  008)2/11.  29  ■  .  5130 

For  S  r.  0.  10,  table  IV  gives  X-  ).  357;  K  *  1.  324.  Since  k>  K,  0.  81 
etrcsi-atrength  reliability  ie  demonstrated  at  the  90%  confidence  level. 
Similar  calculations  show  that  at  the  50%  confidence  level  (use  £  =  .  50, 
table  IV),  the  demonstrated  R  ie  0.96. 
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In  the  event  that  £  >  9 ,  the  interpolation  procedure  for  X  may  be 
accomplished  by  interpolating  on  first,  then  on  12/VT  For  example, 
suppose  i  -  15,  0,  *  0,  8251.  For  90%  confidence,  find  £  »  0,  10  in 

table  IV.  Then  interpolate  in  table  IV  as  follows; 


12/rf* 

4 

3.  873 

3 

■nA.f 

9 

15,  0 

it  1 

0.  8 

1.35lS~ 

1.3380 

,  8251 

i.  3538 

1.  351 

1, 3395 

0.9  1 

1.  3540 

1. 3398 

After  linear  interpolation  based  on  fl,  the  horizontal  interpolation  is 
based  on  12//T,  resulting  in  X.  s  1 .  351. 

Case  3;  Equal  or  Unequal  sample  Sizes,  b  a  CT ^1  0“<g^  Known, 

7  7 

If  the  ratio  of  variances,  O’  ^  /  Cfgg  is  known,  but  the  level  of  each 

variance  is  unknown,  the  procedure  for  finding  A  is  the  same  as  in  cases  1 
It  2.  In  other  words,  equation  (1)  and  the  same  sample  values  of 

Slt,  »th<  ntU'  *n^  n*s  8ive  A  ■  1.  843  as  before, 


If  we  also  assume,  however,  that  the  ratio  of  population  variances, 
““^th  K  may  bo  determined*  as  follows; 


iv) 


<bnth>*  »I? 

"th-1  n..-‘ 


(10  +  9)Z 

T  T 


13.  82  , 


Then 


*  If  n^  =  nifl  =  n,  K  is  more  easily  found  by  solving  equation  (2),  sub¬ 
stituting  O'  ^  for  s^  Rnd  O’ for  »§i>  then  entering  table  III  with 

effective  sample  sire  as  explained  in  rase  1. 


ns 

(8) 
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S  a 


nth  nla 


For  R  b  .99.  ZR  =  2.  226 


/  4  »  1 

6  «  2.  326  /  TTT  a  5.  450  , 

V  5  9 


Using  equation  (5), 

5,450/ 

^  ■  /manor"^  *  ■?2°i  • 

V  27,64 


Linear  interpolation  on  /[,  with  interpolation  on  12/-/7(*ince  f  /  9)  in 
table  IV  (  €  -0.10)  gives! 

K  Si  4,  65 


usinR  ?qu»tion  (6),  As  before  ft  <  K,  and  R  »  ,  99  is  not  demonstrated. 

By  trie,  and  error  we  may  reduce  R  and  the  corresponding  ZR  suffi¬ 
ciently  to  find  Uioi  fui  R  >  .  85,  Zp  =  1, 036  and  K  *  l .  Ibb.  Since 
K  >  K,  R  a  ,  85  is  demonstrated  it  the  9C%  confidence  level.  Similarly, 

R  b  .  96  is  demonstrated  at  tire  DO  %  confidence  level. 

2  2 

Cao«  4.  Equal  or  Unequal  Sample  Slees,  Both  U"^  and  ^  lie  K^own. 

2.  ? 

If  the  level  of  each  population  variance,  and  r'-  6  la  known  separately, 

sampling  variation  is  due  only  to  xth  and  1Tgg  ,  the  sample  means.  Thun 
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}  19 


*th  ■ 


A* suming  that  7  «  4.  00,  Cr,,2  ■  2.  00,  «  13,  28,  Sgi  ■  7.  22, 


13,  26  -  7.  22 


K" /4.00+  2.00  *2,47' 


Compute  K  from!  qZ^ 

(10)  K  »  lA  +  Zc  '"th  PaT 


tr  1  th 


Z  2 

(obtained  by  solving  equation  (15)  after  substituting  ff'th  end  £r>(( 
lor  etft  end  ag2  ,  respectively). 

Where  Z„  end  Z_  ere  critical  value*  of  the  atandard  normal  deviate 
«  C 

••xcooded  by  1  -  R  and  1  -  C,  respectively,  found  in  a  table  of  the  cumulative 
normal  distribution.  Annumlng  R  >  ,  99  and  C  *  .  40,  Jien  Z^  >2.326 
and  Zq  u  1,  ZH2  (found  in  u  normal  distribution  table  such  as  reference  9). 


K  ■  2.  326  -b  1.  282 


1  +  i 

5  9 


a  2.  510. 


Since  &<  K,  R  “  ,991*  not  demonstrated  et  the  90%  confidence  level. 
Accordingly  Z_  (end  therefore  K)  mey  be  reduced  to  1. 925  (R  »>  .  973), 
ueing  equation  (10)  such  that  K  ■  ft  *  2.  46.  Then  R  •  .  973  is  demon¬ 
strated  at  the  90%  confidence  level.  In  eimilar  manner,  R  *  .  993  le 
demonstrated  at  the  50%  confidence  level, 
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PRECISION  AND  ACCURACY  OF  K,  Precision  in  the  critical  value 
of  K  is  dependent  on  whether  the  ratio  of  variances  (or  standard  deviations) 
of  stress  and  strength  is  known,  If  this  ratio  is  unknown!  as  in  cases  1  and 

2  above,  die  precision  is  largely  dependent  upon  the  sample  sices  used  in 

estimating  the  variances  of  stress  and  strength,  measured  by  the  effective 
degrees  of  freedom,  f.  If  die  ratio  is  known,  precision  is  as  good  as  the 
error  in  tables  III  and  IV,  Accuracy  of  the  critical  value  of  K  is  a  function 
of  effective  degrees  of  freedom,  f,  of  the  approximated  ^(2ii.  a.  ,  bias 
decreases  as  f  increases.  It  should  be  noted  that  K  is  a  function  of  the 
ratio  of  <T\.a2  b,  not  their  levels, 

If  the  levels  of  each  population  variance  are  also  known,  equation  (10), 
case  4  above  shows  that  X  is  precisely  and  accurately  determined  (no 
error,  within  limits  ol  the  tabulated  cumulative  normal  dlatribution).  Howeve 
such  knowledge  ie  rare  in  practice.  The  remainder  of  thie  discussion  will 
therefore  be  limited  to  cases  1,  2  (unknown  ratios  of  variances),  and  caie 

3  (known  ratio  of  variance*)  above. 

ACCURACY  FOR  KNOWN  RATIO  OF  VARIANCES  (OR  STANDARD 
DEVIATIONS).  When  the  ratio  b  ■  •r^l  CT  ,Vfl  ie  known,  tilae  in  the 

critical  value  of  K,  due  to  biae  in  f  and^z,  may  be  found  by  comparing 
critical  value*  of  K  as  obtained  in  case  3  above  with  the  more  accurate 
value  of  K  obtained  by  Monte-Carlo  simulation  of  equation  (1)  (see  referen¬ 
ce  I,}.  In  illustration,  examine  table  I; 
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TABLE  I 

ACCURACY  OF  K,  KNOWN  RATIO  OF  VARIANCES 
(OR  STANDARD  DEVIATIONS)2 
R  «  .99 


b"  ^th/^ss2 

"th 

Bss 

l 

C  ■  ,  90  ■  .  50 

K,  Mont* -Carlo 

2000  Simulations 

C  ■  .  90  C  ■  .  50 

1 

5 

5 

8.0 

3.64  2.421 

3.  79, 

2.  40 

1 

7 

7 

12.  0 

3.31  2.389 

3.  45, 

2.40 

1 

9 

9 

16 

3.  14  2.  372 

3.22, 

2.  34 

1 

20 

20 

38.  0 

2.83  2.35 

2.85, 

2.  35 

1 

5 

9 

12.  0 

3,47  2,40 

3.48, 

2.406 

1 

9 

5 

12.0 

3.47  2,40 

3.47, 

2.405 

3 

S 

9 

8.  7# 

3.  67*  2.  44* 

3.  78, 

2.44 

3 

7 

7 

9.6 

3.53  2.41 

(  3.  50, 
\3.  54, 

2.43'l 

2  44/  (rep,  > 

3 

9 

5 

10.  5 

3.  39  2.40 

3.  41, 

2.  42 

3 

20 

20 

30.4 

2.  87  33 

2.  89, 

2.  33 

9 

3 

9 

5.  6 

4.21  2.52 

4,  22, 

2.  .03 

9 

7 

7 

7.  5 

3.67  2,45 

3.  69, 

2.45 

1 

50 

50 

98.  0 

2.  64  2.  U 

2.  64, 

2.  32 

Comparison  it  with  Monte -Carlo  solutions  indicates  that  when  b  ;>9 

sample 

eieee  for  which  f  £  S  give  very  high  accuracy;  when  b>  1,  f  >20  gives 
very  high  accuracy  of  the  analytical  solutions; 


*  Theie  result*  are  based  on  data  of  figure  I  if  s,  should  happen  to  equal 

0^2. 
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PRECISION  OF  K  FOR  KNOWN  RATIO  OF  VARIANCES  (OR  STANDARD 
DEVIATIONS),  When  the  ratio  b  =  &  i»  known,  f,  &  ,  Hi  and 

K  in  eaee  3  are  uniquely  determined,  within  tha  precliion  of  table  III  and  IV, 
This  is  readily  leer,  by  examination  of  equation*  (7),  (B),  (5),  and  (6), 

PRECISION  OF  K  FOR  UNKNOWN  OR  ASSUMED  RATIO  OF  VARIANCES 
(OR  STANDARD  DEVIATIONS),  In  the  abience  of  knowledge  of  b  in  the 
relationship  o'  £  ■  b ,  lack  of  precision  in  K  is  due  to  the  use  of  *j2, 

the  sample  estimate*  of  C T f  in  equation  (2)  in  case  1,  and  (3)  and  (4)  in 

case  2,  K  was  alto  obtained  through  Monts-Carlo  simulation  of  equations 
(2)  through  (6),  obtaining  2000  estimate*  of  K  as  a  function  of  2000  random 
values  of  both  st^z  and  sla2>  where  and  xBa  were  sampled  randomly 
from  normal  distribution*  with  variances  Cy ^  and  CT,,2  ,  respectively, 

Table  II  below  illustrates  the  over-all  precision  of  the  method  fur  the  case 
R  a  .  99,  C  «  ,  90.  It  is  of  interest  to  examine  equation  (7),  Table  II  or 
Figure  III  regarding  l  vs,  K,  For  n^  +  naa  constant  and  the  assumption 

b>l,  f  Increases  as  n^/n,,,  decrease*.  Conversely,  under  the  assumption 
b  <  1,  f  increase*  a*  nth/nfg  increases.  Further,  under  all  of  these  con¬ 
ditions  that  increase  f,  K  decreases,  permitting  demonstration  of  reli¬ 
ability  at  a  higher  confidence  level  (or  higher  reliability  at  fixed  confidence). 


TABLE  XI 

PRECISION  POR  UNKNOWN  VARIANCE  RATIOS 


2 

<r  th 

2 

AT** 

nth 

n*. 

£* 

Oantral  90 
Percentile* 
of  £ 

Central  90 
Percentile* 
of  K 

5%  937. 

&  sn 

9 

1 

5 

9 

3.6 

4.44,11.16 

3.96 

9 

1 

7 

7 

7.5 

6.32,10.53 

3.46,3*91 

3 

l 

5 

9 

fl.679 

3.26,  11.94 

-**,  3.96 

3 

1 

7 

7 

9.600 

6.98,11.94 

6.90,11.93 

3.39,3.80 
(rofi.)  3.39,3.81 

3 

1 

9 

5 

10.316 

8.52,11.98 

3.40,3.59 

3 

1 

20 

20 

30.401 

24.47,37.06 

2.83,2.96 

3 

1 

13 

3 

16.368 

14.48,17.93 

3.17,3.20 

3 

1 

23 

13 

32.342 

27.84,37.18 

2.83,2.91 

3 

1 

30 

10 

34.433 

31.18,37.64 

2.87,2.88 

l 

l 

3 

9 

11.973 

7.87,12.00 

3.39,3.70 

- 

1 

7 

7 

12.000 

7.96,12.00 

3.38,3.66 

1 

1 

9 

5 

11.973 

8.00,12.00 

3.39,3.67 

l 

1 

20 

38.000 

32.22,38.00 

2.83,2.38 

l. 

1 

15 

5 

17.921 

12.17,18.00 

3.17,5. 

1 

1 

25 

13 

37.996 

31.79,38.00 

2.65,2.90 

1 

1 

30 

10 

37.969 

ji. 93, 38. 00 

2.87,2.9' 

1 

1 

SO 

:o 

98.000 

90.96,98.00 

2.63,2.64 

TTor” 

Known 

0\b2/  rl"»* 

w*FiEth  p*rc*ntile  of  f  Ulow  lower  limit  o£  £  “ 


6  in  Table  IV 
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Table  11  Indicate*  that  in  the  absence  of  knowledge  ol  b  »  .  • 

minimum  value  ol  £  o £  approximately  20  1*  requirod  tor  reasonable  pre- 
cielon  o£  the  90%  confidence  e»tlmate  o£  R. 


Figure  III  indicate*  what  ratio  of  *ample  *i*e*.  a  *  nth  /ngg  mlBht  bo 
selected  to  obtain  large  £  and  therefor#  high  preciiion  o£  K  a*  a  function 
of  b  u  ^th2/  and  n#  .  The  curve*  in  figure  III  were  plotted  by 

putting  equation  (4)  in  the  forms 


(ID 


(*b  + 

“  ]*£Er 


an 


•  •-1 


A  *  1 

nBS  1 


Examination  of  thi*  equation  ihow*  the  following  limiting  characteristic* 
when  ngg  >  1: 

A*  b  — *  o,  £“4n»»  “ 

b — loo,  f — >nth  -  1 

a— *0,  r“”*ne*  '  1 

a — 1*0,  , 

Aleoi  for  a  ■  1 ,  l.  e. ,  ntn  ■  ngg  «  n,  b  -  1  give*  a  maximum  value  of 
f  ■  2(n  -  1). 

It  i*  of  interest  to  note  that  in  figure  III,  if  we  can  munsf  only  that 
b  -  <vth2/  *gg2  >1  thon  eelection  of  n^/n,,  j.  1  give*  fctrly  coneiete-: 

level*  of  f,  and  the^eivve  con*i»tent  precision  for  specified  value*  of 
no,  and  n  due  to  neax  -iiatr.e**  of  the  curv*  in  that  region,  Conversely 

wi  f»«  7  7 

if  | rgl2/  <rth2>l»  lt  1®  advantageous  to  delect  ngg  /n^-'  >  1. 

Tabl«  u  ihow*  the  5th  end  95th  percentile*  of  K  for  various  com¬ 
bination*  of  or2  ^'B2,  nth  and  ngg,  with  corresponding  f.  Fora 
ratio  s^2/*,,*,  combinations  of  ny,  and  ngg  may  be  selected  on 
the  basis  of  *mall  differences  of  the  5th  and  95th  percentile*. 
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If  *uch  sample  are  prohibitive,  a  poMlble  solution  to  the  problem 

would  be  to  determine  ratio*  of  variance*  of  strength  and  *tre«*dn  the  ba*l* 
of  experience.  Determination  of  level*  of  each  variance  t*  unnecessary  if 
the  ratio*  are  known  precisely.  Then  dependent  upon  f,  n^,  ng#,  and  b, 
table  I  would  indicate  whether  or  not  to  uae  the  Monte «Carlo  oir  analytical 
•olution  of  K,  ae  a  function  of  the  permiaiible  amount  of  bias  in  the  analytic 
cal  method.  A*  mentioned  in  reference  (3),  thl*  bi*»  vanishes  if  one  of  the 
variance*  1*  overwhelmingly  larger  than  the  other. 

Appendix  B 

DERIVATION  03*  TUB  METHOD.  Derate  xiu  a*  strength,  and  x#g  as 
•tree*,  For  brevity' •  sake,  define  the  probability  that  *tre*«  1*  lei  a  than 
strength  a*  "streas'-etrength  reliability,  "  R.  Then 

(1Z) 

R  -  P(xM<  xth) 

*  P<*th  *  >  °>* 


Assuming  that  l»  distributed  normally  with  mean  and  standard 

deviation  0*^  that  x#g  it  di»trlbu ted  normally  with  m*an  /XB<  and 
■  tandard  deviation  d^g,  and  that  xt^  and  xgg  are  independent,  equation 
(12)  vney  be  written  a*  (reference  6): 


(13) 


R  » 


VTrr 


^Th 


i  r 

TTtfJo  • 


-  [xth"*»*~(  M'th'  ^h«)] 


Z  (O' 


th 


O'..4) 


.d(xth  >  xgi) , 


Till*  integrel  in  shown  in  figure  IVi 
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In  other  word*  equation  (13)  state*  that  x^  ■>  xgg  is  normally  dis¬ 
tributed  with  mean  jU^  -  M',,  end  standard  deviation  Vcr"^  +  <X . 

Actual  *tre*i-*trength  reliability  ie  a  function  of  the  number,  ZR,  of 
such  itandard  deviation*  between  0  and  the  mean  (M'th*  Clearly, 

the  more  etandard  deviation*  by  which  |U>th  -  ptgg  exceed*  0,  the  higher 
the  etreei-etrangth  reliability.  When  tho  population  mean*  and  etandard 
deviation*  are  known,  etreee-atrength  reliability  ie  determined  (100%  con  ¬ 
fidence)  from 


(»■») 


Z 


r.  “ 


^th  ' 


rr 

*■ 


i*.i  which  caee  R  la  obtained  from  ZR  (one-iided)  ueing  a  table  of  the 
normal  distribution 

Unfortunately,  we  eeldom  know  the  population  mean*  uud  itandard 
dn-'iition*  for  u*e  In  equation  (14).  In  *uch  cases,  we  can  only  cellmate 
from  samples  (but  not  lei*  than,  *ay,  S)  of  %  “*•••  the  eample  eitlmate 

of  the  nume r»tor  of  equation  ( 14) ,  and ~\J*^  +  *g()  >  the  (ample  estimate 
of  the  denominator  (x,  the  sample  mean,  denote*  estimate  of  l  s,  the 
sample  standard  deviation,  denote*  estimate  of  9“  )  Using  such  sample 
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ditii  a  procedure  for  demonstrating  stress-strength  reliability  at  a  given 
confidence  level  1*  a*  follows;  Find  a  constant,  K,  large  endugh  that 
xlh  -  -  K-^sj^  +  •**  ^•l*  tl',n  tkl*  propo*^011  R  the  population  of 

strength-minus-stres*  vatuos  100C%  of  the  time.  This  may  be  stated 
mathematically  as 


0  5)  P(xth  -  x„  -  K^hV.2„  £  Mth’  A„*  ^<4  + 

Then,  if  3cth  -  x|§  -  K +  **s*  ‘‘“liability  t>f  at  least  R  is 
demonstrated  at  confidence  C  (figure  V),  Or,  if  £,  »  (*th'^a  s>/>X.» 
at  loast  R  is  demonstrated  at  confidence  C, 
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The  approach  given  herein  lor  finding  K  it  an  application  of  theory 
developed  by  3.  L.  Welch  and  also  investigated-  by  Alice  A.  A  spin. 

Mr.  Welch  developed  a  atatietic  approximately  dietributed  as-  “X  £ 
(reference  3,  p.  31)$  which  may  be  applied  ae  follows  in  the  bivariate 


2 

*«  +  Xj**  j 

|  B  *1 

V  *.  \  =t 

<y 

Then  s/g  ie  approximately  distributed  an  "X  with  f  degrees  of  freedom 
where  A.j  and  ar*  constant*  and 

t  M  *i8  ♦  *a  fa V 


*JjL  x  'Vl*  +  *2*22 

{  "  x*i5r;~w 


<% 

is  approximately  distributed  as  X  / f.  In  a  later  papa (refe  retvie  4, 
p.  ItJ),  Mr.  Welch  indicate*  that  in  the  absence  of  knnwledge  of  0-j,  l 
may  he  estimated  by  replacing  each  9}  in  aquation  (18)  with  its  estimate 

Y 


(181 


(IS! 

Thus 


case: 

(16) 

Let 


(17) 


The  approximate  non -central  t  statistic  utilised  herein  is  defined  in 
the  usual  manner,  substituting  y.£/t  as  def.ned  above. 
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SOLUTION  OF  K,  EQUAL  SAMPLE  SIZES,  KNOWN  OH  ESTIMATED 
RATIOS  (b)  OF  VARIANCES.  If  the  number  of  stress  measurements,  n, 
is  equal  to  the  number  of  strength  measurements,  equation  (15)  may  be 
written  as: 


(20) 

P 


/ 


+  ^ 


\ 


6k\^“  >  ■  c, 


) 


By  the  method  of  reference  3,  equations  (23)  -  (26),  the  left  side  of  the 
inequality  is  approximately  the  non-central  t  statistic  with  parameter  of 
non-centrality  ZR*J"n,  which  is  the  same  parameter  used  in  calculating 
tables  of  one-sided  tolerance  factors  for  normal  distributions  (bold-face 
values,  table  II  of  reference  5),  Also,  becaueo  the  right  Side  of  the  in¬ 
equality  is  the  same,  K  is  obtained  directly  from  such  a  table,  (table  III) 
us  a  function  of  R,  C,  and  effective  sample  sise.  (Table  III  herein  la 
the  same  as  the  bold-face  values  of  table  II,  reference  5,  but  more  com¬ 
plete).  Subetttuting  Ai  -  1  and  ft  ■  n  -  1  in  t  quatlon  (18)  above: 

(21)  f~  '.I  -  i)  t  l  <r\f 

+  o-S, 


Which  may  be  estimated  us  described  in  the  narrative  following  aquatic., 
(19)  as- 


In 

— 


+  s2  )2 

ss' 


s 


4 

~.s 


(22) 


T 

■th 


+ 
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In  the  absence  of  Knowledge  of  b-  ^ I  O'..2'  the  effective  ssmple 
else  for  entry  in  table  111  (taken  by  permission  from  reference  B)  is 
f  +  Is 

(23)  (n  -  1)  (•**  +  •ss2*2  4.  j 

Effective  sample  sise*>  -  i  -  — — —  • 

•th  +  *ss* 


It  may  be  noted  that  for  th.  case  ^  -  b  the  effective  sample 

else  is  independent  of  the  standard  deviation.  In  that  case,  Effective 
Sample  Sise  as  1  f  (n  -  1)  (b  +  l)r/*» 

snt  AJTION  OF  K.  SAMPLE  SIZES.  KNOWN  OR  ESTIMATED 

n  a-TTS’ Thfor  VARIANCES.  If  the  number  of  stress  measurements,  ... 
to.,  not  equal'  the  number^!  strength  measurements  n*.  equation  (15) 

may  be  written  asi  ' 

f  *th  *  ‘  <  M-th-  ^aa>  +  •  C 


+  n 
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By  definition  of  non-central  t,  the  quantity  In  bracket*  on  the  left  »lde  of 
the, inequality  of  equation  (23)  it  approximately  the  non-central  t  statistic 
with  parameter  of  noa-centraUcyt 


For  R  ^  0.  30,  Z.R  is  the  positive  one-eided  standard  normal  deviate 
tor  responding  to  R.  u  0-^2  .  bcr,,2,  equation  (26)  becomes: 
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The  effective  digrui  of  frtcdom.  f,  may  also  be  obtained  in  the 


manner  of  oquatlon  (18),  where  X  j 


"  ni' 


2\2 


(27) 


lnch  **  th*  +  nea 

V  -  M  +  ai'fi Ku-  u 


which  may  bo  estimated  by: 


(28) 


<«th« 


th"th 


+  n..*B.2)Z 


*  nth2,th*^rtth  *  +  n»«*,M^^(n»»  “  2) 


A»  and  na>  become  large  tha  approximation  to  tha  non -central  t 
distribution  improves, 

2  2 

In  tha  event  that  <5V  "  ^.a  •  aquation  (27)  become*  independent 
of  tha  etandard  deviation: 


(2  BA) 


f  = 


(b»th  + 

ibVt 


nth-1 


t 


Tha  critical  non  ctmral  t  value  la  now  obtained  from  table  IV,  a* 
a  function  of  o  of  equation  (381  and  f  of  equation  (28),  by  tha  method  of 
reference  7  a*  follow  a: 


First,  compute 
(29) 


StJlT 
\Jl  *  $lm 


Second,  find  A.  in  table  II  at  a  function  of  Th  ,  f,  and  confidence 
Cal  -  £  .  Turn  to  the  pago  corresponding  to  the  appropriate  value 
of  £  .  Then  interpolate  linearly  between  valuta  of  yj,.  and  linearly  bo- 
tween  value*  of  f  whenf<9.  When  f>9>  Interpolate  on  12 //T^atwevn 
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values  of  f.  For  confidence  levels  between  .  50  and  .95  and  reliability 
levels  of .  30  or  more,  use  only  positive  values  of  f0  (tee  reference  7). 
The  critical  ..jn-central  t  exceeded  by  £  »  1  -  C  lei 


<3°)  t  -  Six'll i  +  -  X»/»T 

l“e  i  -  jFni 


And,  as  indicated  by  equations  (25)  and  (26),  the  critical  value  of  K 
exceeded  by  1  -  C  1st 


(31) 


K  SSt 


tl  -  6 

f /ZR 


(Note  that  **  tbs  radical  in  equation  (26),  which  has  been  com* 

putsd  in  the  process  of  calculation). 
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'  STATISTICAL  TESTING  TECHNIQUES  USED  IN  THE  DEVELOPMENT 

OF  T  HE  PRATT  k  WHITNEY  AIRCRAFT  RL  10  ROCKET  ENQ1NE 
,  FOR  THE  NATIONAL  AERONAUTICS  AND  SPACE  ADMINISTRATION 

Harold  J.  Tiedemann 
Pratt  tt  Whitney  Aircraft 
Florida  Reaearch  and  Development  Center 

INTRODUCTION,  Pratt  k  Whitney  Aircraft'*  experience  In  the  de¬ 
velopment  of  many  different  kind*  of  propuleion  eyeteme  haa  ehown  clearly 
the  necessity  nf  a  comprehenelve  development  program  of  three  to  five 
yeara  to  bring  a  new  power  plant  from  preliminary  dealgn  to  operational 
capability.  Reliable  operation  of  luch  a  eyetem  ie  achieved  by  Integrated 
rnd  experienced  engineering  daaign  and  development  team*,  proven 
manufacturing  method*,  rigid  quality  and  vendor  control,  and  a  eupportlng 
field  service.  Emphaeii  muet  be  placed  on  a  comprehenelve  teat  program 
involving  perhaps  thousands  of  tests  of  both  individual  components  and 
full-eoaie  engines.  There  ie  no  means  to  reallie  the  full  potential  and 
reliable  operation  of  a  power  plant  or  eyetem  except  by  a  comprehenelve 
development  program  which  involve*  many  cycle*  of  design,  testing, 
corrective  design,  and  retesting,  The  element  of  time  cannot  be  factored 
out  ot  this  development  cycle, 

During  the  course  of  most  euoh  development  programs  a  number  of 
complex  problem*  arise  where  the  time  involved  in  test  evaluation  can 
be  minimised  and  morn  complete  information  obtained  by  the  use  of 
statistical  testing  techniques,  Several  representative  examples  of  .hcee 
testing  technique*  have  been  used  in  the  RL10  Rocket  Engine  Develop¬ 
ment  Program  being  conducted  for  the  NASA  -  Marshall  Space  Flighi 
Center,  These  include  Full  Factorials,  Latin  Sou* ro»,  And  Random  and 
Multiple  Dslance  technique*.  The  purpose  of  this  paper  is  to  discuss 
these  technique*  in  detail,  pointing  out  their  best  area*  of  spollcatlon 
*.»  well  as  their  Advantages  and  disadvantages,  and  to  describe  levt.’1 
RL10  engine  development  problem*  which  were  successfully  solved  by 
the  use  of  these  methods, 

These  Include  a  Full  Factorial  to  determine  thrust  control  settlr* 
repeatability,  and  a  partial  Full  Factorial  to  provide  additional  fuel 
pump  (tail  margin,  Also  included  is  the  description  of  a  Latin  Square 
teat  program  designed  to  determine  the  principal  factors  in  controlling 
transient  thrust  overshoot.  Finally,  a  Random  Balance  test  ie  described 
which  was  used  to  determine  method*  to  control  thrust  control  hvetereeie 
and  non-iinearlty, 
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STATISTICAL  TESTING  TECHNIQUES, 

A.  THE  FULL  FACTORIAL  TEST, 

In  the  full  Factorial  tatting  technique,  teitt  ere  made  with  *11  pottible 
combination*  of  the  independent  variable*)  therefore,  all  Interaction*  be¬ 
tween  parameter*  will  be  identified,  Thua,  a  Full  Factorial  might  be 
described  a*  a  Cully  expanded  version  of  the  *tep-by-*tep  approach,  A 
Full  Factorial  experiment  i*  illustrated  a*  follow*) 

Suppose  that  there  are  three  independent  variable*,  each  at  two  level*  , 
of  which  either  the  main  effect  of  a  single  variable  or  the  interaction  of 
two,  or  even  three,  of  the  variables  ha*  an  effect  on  the  result,  The 
three  independent  variable*  are  denoted  by  A,  B,  and  C  and  the  level*  by 
the  subscript*  1  and  2,  The  number  of  possible  combination*  i*  2®  ■  8, 

A  matrix,  or  block  test-plan  representation  of  the  Full  Factorial  test  i* 
given  a*  follow*) 


B 

B 

B 

B 

1 

2 

1 

2 

C1 

C1 

C2 

C2 

°1 

°1 

C2 

C2 

Trrre  *r«  several  advantage*  In  running  a  Full-Factorlal-deslgnec* 
test  program.  These  Include  complete  information  on  individual  main 
effect*  a*  wall  ae  complete  Information  on  ail  Interactions,  Furthermore, 
for  the  same  number  of  tests,  the  Full  Factorial  approach  give*  a  more 
preciu*  estimate  of  the  main  effects  than  the  etep-by-etep  approach.  The 
Full  Factorial  <«  the  only  type  of  designed  experiment  that  will  positively 
identify  all  Interactions  among  the  variablee  In  the  experiment,  none  of 
which  may  be  identified  by  the  stop-by  -step  approach. 

The  tests  muet  be  run  in  a  random  or  near-random  manner  to  avoid 
distorting  the  result* ,  An  effect  might  be  faleely  identified  ae  significant 
if  runs  at  the  low  level  of  a  variable  were  made  at  one  time  and  the  rune  at 
the  high  level  were  made  later.  A  shift  in  the  measured  output  from  the 
first  to  the  second  time  period  could  be  caused  by  any  factor  changing  with 
time,  Occasionally  It  may  appear  impossible  to  finish  a  Full  Factorial, or  some 
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suspicion  ivi*y  exist  that  a  main  effect  of  a  single  parameter  rather  than 
the  interaction  of  several  parameters  is  the  true  reason  for  a  given  result. 

If  soi  it  may  he  more  judicious  to  run  the  first  few  tests  in  a  predetermin¬ 
ed  sequence  to  gain  the  greatest  preliminary  information, 

Full-Factorial-daelgned  experiments  have  been  used  many  times  in 
the  development  of  the  RL  10)  this  report  will  discuss  two  types  of  Full 
Factorial  experiments.  The  first  of  theeei  whlcn  is  designated  Full 
Factorial  A,  concerns  a  test  mads  to  determine  the  rather  complex  inter¬ 
relationships  that  exist  between  engines  and  thrust  controls.  The  second) 
which  is  designated  Full  Factorial  B,  concerns  an  analog  program  to  deter¬ 
mine  the  important  parameters  in  a  fuel  pump  stall  situation.  Although 
not  strictly  an  engine  test  program,  Full  Factorial  B  represents  the  use 
of  e  limited  Full  Factorial  experiment  in  the  analysis  of  several  para¬ 
meter*  to  evaluate  their  contribution  to  fuel  pump  stall. 

1.  FULL  FACTORIAL  A. 

The  Full  Factorial  A  tsst  program  was  run  to  determine  the  thrust 
control  setting  repeatability  from  angina -to -angina  and  to  show  the  effect 
of  running  a  aeries  of  preset  thrust  controls  on  e  given  single  engine. 

In  this  manner  Individual  control  and  engine  characteristics  were  demon¬ 
strated  independently.  Ae  e  further  variable,  both  hot  and  cold  thrust 
control  housings  and  thrust  chambers  ware  run,  A  representation  of  the 
plan  la  given  below.  The  independent  variables  ware  engine  number,  thrust 
control  number,  end  thruet  control  end  chamber  temperatures. 


Tluusi  Control  A 

Thrust  Control  B 

Thrust  Control  C 

Ermine 

Trim  Repeat  Repeat 

Hot  Hot  Cold 

Repeat  Repeat 

Hot  Cold 

Repeat  Repeat 

Hot.  er>ld 

Engine 

_ u _ 

«♦  II  II 

II  II 

M  II 

The  Full  Factorial  test  program  was  justified  because  both  individual 
engine  and  thrust  control  effects  on  sngino  trim  setting  repeatability  had 
to  b«  known  so  that  specific  step*  could  bn  taken  to  improve  trim  setting 
end  trim  repeatability.  Also,  it  was  known  that  a  greet  deal  of  corollary 
information  of  the  effect  of  thrust  control  characteristics  on  other  engine 
performance  parameter  a  would  be  made  available. 
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In  this  td •  t ,  both  angina*  '.vara  trimmod  to  rated  thruat  conditions  with 
Thrust  control  A  on  tha  first  run  of  th*  sariai  and  two  repeat  run*  made 
with  tha  Lama  thruat  control.  Thruat  Control  A  was  than  banch  calibrated 
and  Thrust  Controls  8  and  C  were  sat  to  this  calibration.  Tha  remaining 
runs  of  th*  plan  war*  then  made.  The  affect  that  each  thruat  control  setting 
had  on  individual  run-to-rim  trim  repeatability  and  engine -to-engina  re* 
paatability  was  demonstrated  in  this  manner;  assuming  that  all  other 
influences  ware  known  or  approximated.  From  this  test,  it  was  shown 
that  custom  ssttlng  of  thrust  controls  appears  to  be  feasible  although  tren- 
slant  thruat  control  psrformanc*  and  steady  state  repeatability  heed 
further  definition. 

Additional  conclusions  are  that  the  trim  angina  propellant  mixture 
ratio  ha*  eat  stabilisation  time  and  ie  apparently  independent  of  atart  in¬ 
let  temperature.  However,  stabilisation  time  for  trim  thruat  la  indicated 
at  different  values  for  a  chamber  cold  etart  than  for  a  chamber  hot  start, 
Other  important  conclusions  on  the  effect  of  control  characteristics  on  thrust 
overshoot,  rats  of  acceleration,  stability,  and  repeatability  resulted  from 
this  test.  Thruet  control  bench  techniquee  required'  to  preeet  thruat  con¬ 
trols  for  field  operation  were  determined. 

2.  FULL  FACTORIAL  B 

Early  in  thu  RL  10  engine  development  program  11  was  apparent  that 
fuel  pump  stall  wee  an  occasional  phenomenon  of  the  engine;  its  occuroncc 
was  dependent  on  several  environmental  and  operational  factor*.  Test 
experience  and  analytical  studies  indicated  that  at  least  15  of  these  factors 
appeared  to  have  a  bearing  on  pump  gta.ll.  Obviously,  even  if  d,e  number 
of  values  per  parameter  were  limited  to  just  two,  the  number  of  possible 
combination*  of  all  of  these  factor*  in  a  Full  Factorial  would  he  a  large 
number  of  test  run*  (2^)  or  a  large  number  of  analog  machine  run*  (a* 
it  was  In  this  case). 

The  15  factors  that  moil  affected  pump  stall  were: 

1,  Fuel  pump  bleed  valve  area;  pump  interstage  and  discharge  bleed 
valves  am  used  on  the  RL10. 

2.  Fuel  pump  bleed  valve  closing  schedule;  that  is,  the  point*  in  th* 

start  transient  where  the  bleed  valve  starts  to  close  and  where 
it  finishes  closing. 
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3.  Fuel  pump  blade  characteristics!  that  la,  the  amount  of  aweep 
In  the  first  and  eecond  atagea. 

4.  The  fuel  pump  inlet  preeaura  at  the  start  signal  and  throughout 

the  traneient. 

5.  Engine  thrust  control  gain  --  ueually  axpraaaad  in  terma  of  pai 

of  chamber  praaaura  per  unit  of  mixture  ratio  variation. 

6.  The  maximum  thruat  control  bypaaa  arua  that  opens  near  the  top 

of  the  atart  tranaiant  to  limit  angina  r.verahoot  by  paaalng  fuel 
around  the  propellant  drive  turbine. 

7.  The  Lox  pump  praaaura  at  which  the  mixture  ratio  valve  on  the 

Lox  aide  of  the  engine  opana. 

8.  Lox  pump  inlet  praaaura  at  atart  and  during  the  start  tranaiant. 

9.  The  jacket  metal  temperature  that  determines  the  amount  of 

energy  imparted  to  (ha  turbine  drive  of  the  RLIO. 

10.  Venturi  area  upstream  of  the  fuel  turbine. 

11.  Turbine  to  venturi  area  ratio. 

12.  Fuel  pump  discharge  orifice  diameter. 

13.  Turbine  efficiency, 

14.  Fuel  pump  efficiency. 

15.  The  amount  of  time  the  thruit  control  ie  open  during  the  uve.. 

■hoot  control  period. 

To  reduce  the  number  of  runr  required  to  complete  a  Full  Factorial 
tost  on  these  parameters,  it  wa*  decided  to  concentrate  only  on  those 
factors  that  were  coneidered  moat  important  for  the  results  of  tne 
experiment.  Based  on  available  experience,  these  were  (1)  fuel  pump 
discharge  orifice  diameter,  (2)  fuel-aide  venturi  area,  and  (3)  the  ratio 
of  turbine  area  to  venturi  area. 
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A  Full  Factorial  experiment  of  24  rum  wa»  made  as  the  firat  part  of 
the  analog  program  with  four  pump  discharge  orifice  diameter  voluee, 
threo  venturi  area  value*,  and  two  value*  of  the  ratio  of  turbine  area  to 
venturi  area.  Because  this  was  an  analog  program,  there  wua  no  ad¬ 
vantage  in  running  these  teat*  in  a  random  order,  Based  on  the  Full 
Factorial  of  24  runs,  optimum  value*  of  fuel  pump  diachargc  orifice 
diameter,  venturi  area,  and  turbine  vonturi  area  ratio  were  choaen  to 
give  maximum  atall  margin  during  the  start  transient.  Although  these  war* 
the  moat  important  factors  as  indicated  above,  they  were  not  necessarily 
the  only  ones  that,  could  contribute  to  a  atall  altuation, 

The  remainder  nf  the  analog  program  consisted  of  (1)  holding  the 
values  of  fuel  pump  discharge  orifice  diameter,  venturi  area,  and  the  ratio 
of  turbine  to  venturi  area  at  the  values  found  optimum  from  the  24-run  Full 
Factorial  and  (2)  Independently  varying  each  of  the  remaining  parameters 
over  a  two  or  three-value  range.  For  example,  the  ftiel  pump  bleed  valve 
doling  pressure  was  varied  between  100  and  300  peia,  the  fuel  pump 
bleed  areas  were  varied  between  0  and  0.  2  square  inches  for  the  fuel 
pump  discharge  valve,  and  between  0.  2  and  0.  S  square  Inches  for  the 
fuel  pump  interstage  valve.  Similar  variatlone  were  made  in  the  re¬ 
maining  parameters  at  either  two  or  throe  level*  and  Ihs  effect  of  each 
of  these  parameters  was  evaluated.  Baeed  on  this,  e  grsat  deal  of 
valuable  information  was  gained  on  the  effect  of  each  of  these  factors  on 
pump  stall. 

Some  of  the  parameter*  are  not  necessarily  controllable,  For  examplu, 
lurbit  -  and  pump  efficiencies  nr*  not  independent  variable*  but  rather 
are  variables  that  are  dependent  upon  design  con  elds  ration-.  Enough 
information  was  pained  to  greatly  Increase  the  -tall  margin  of  the  RL10 
engine  as  a  result  of  this  and  several  other  experiments  conducted  both 
in  the  unalog  program  and  in  the  actual  test  program. 

B.  THE  luATlN  SQUARE  TEST 

If  there  is  evidence  that  interactions  are  not  significant  belwsen  the 
eevetul  Influencing  parameter*,  and  if  information  of  the  main  affccti. 
of  the  several  parameters  is  what  ie  most  dttired,  a  Latin  Square  test 
program  can  be  set  up  in  which  the  parameter*  are  arranged  a*  illus¬ 
trated  in  the  table  on  the  next  page,  with  three  parameter*. 
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Parameter  1  (a,  b,  e,  d) 
Parameter  2  (A*  B,  0,  □) 
Parameter  3  (1,  21,  III,  IV) 


Here  the  three  parameter*,  lech  at  four  level*,  are  arranged  In 
a  16- teat  experiment,  Not#  that, each, main  effect  of  «ach  of  the  thra* 
parameter*  La  capable  of  being  evaluated  eeparately  by  the  aerie*  of  16 
teat*,  (A  Pull  Factorial  would  require  64  te ate. )  However,  all  inter- 
action*  (if  exiatent)  between  theee  aeveral  parameter*  are  -in  with  : 

the  main  effect*,  The  advantage  of  a  Latin  Square  program  i,«  that  hit  1 
only  are  main  effect*  eitabliehed  quickly  but  they  aha  also  uatabliihed 
with  a  minimum  number  of  run*, 

An  additional  parameter  may  be  identified  with  the  *ame  number  of 
rune  a*  with  the  Latin  Square  by  adding  another  variable  lo  the  Latin' 
Square.  Thl*  1*  known  a*  a  Graeco -Latin  Squares,  adding  twrv  variable* 

(6  total)  would  re»d!‘  a  Hyper-Qraeco-Latin  Square.  The ir  idditlor.el 
eetimatoe  of  main  e.  -euls*  arc  made  at  lh«  expan**  of  thb  preciaipp  of,  the 
estimate  of  experimental  error)  i.e,  ,  thu  total  degree  of  freedom  ia  the 
■ame  for  all  three  deeigne,  and  oar.h  main  effect  require*  a  number  of 
degree*  of  freedom  that  1*  one  lee*  than  the  number  of  Ite  level*  in'  obli¬ 
gated. 

It  ehould  be  pointed  ort  that  in  engineering  problem*  it  t*  difficult 
io  »i.i to  uufoio  in*  experiment,  a  priori,  whether  or  notan  interaction 
may  uxiit.  Many  phenomr...  ha«*  demonetrated  the  exigence  of  lntor- 
nction*.  Interaction*,  »o  aaelly  noglected  in  conventional  engine.  ri-» 
tinalyeui,  can  be  quite  intidioue.  Therefore,  Full  Factorial*  are. 
usually  preferred  to  Latin  Square*,  Fractional  Factorials,  and  ulhur 
"•hort-cut"  *xperim«nt*. 

Thy  following  curv*  demonstrates  the  danger  of  *.ot  considciirg 
inte  r«  HI  one,  ic  the  variable  being  investigated,  and  1«  the  un¬ 

known  variable. 
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While  changing  X1  from  a  low  level  to  it  high  level,  if  the  unknown 
variable  Xu  happen*  to  be  at  a  high  level,  we  get  a  very  slgiiiflcant- 
appearing  main  effect  of  Xp  On  the  other  hand,  if,  while  changing 
from  a  low  laval  to  a  high  level,  Xu  were  oonatantat  it*  low  level,  we 
would  dlicard  Xj  a*  being  completely  unimportant.  Hare  we  have  a 
strong  (X^  x  Xu)  interaction,  with  poaaibl*  moderate  main  effect*  of 
X^  and  Xj.  An  objective  eearch  technique  called  "variation  reeearoh" 

i»  available  for  finding  unknown  variable*  tn  put  into  (tall »tlr ally  de> 
signed  expuiiuio.iU,  This  search  technique  I*  not  described  in  thle 
paper. 

During  the  earlier  phaiei  of  the  RL  10  development  program,  thrust 
overshoot  during  the  engine  etert  tranelent  wee  higher  than  desired 
limits  on  tome  rune.  Three  parameters  that  appeared  to  afiect  over¬ 
shoot  were  thrust  chamber  metal  temperature  (which  determines  tur* 
bine  power  on  acceleration  in  the  RL10  regenerative  cycle),  thruet 
control  time  constant,  and  thvuat  control  body  or  servo  pressure.  It 
wae  necessary  to  ascertain  the  main  effect*  of  each  of  those  parameters 
as  quickly  as  possible  to  determine  where  maximum  development  effort 
■hould  be  expended.  Therefore,  a  simple  four-run  Latin  Square  program 
was  set  up  Since  only  four  runs  were  involved,  the  order  of  tests  was 
not  randomised.  The  program  consisted  of  the  followings 
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Parameter  Level* 


Thrust  control  timo  constant, 

seconds 

0,  30 

0,  55 

Thrust  control  body  pressure 

at  thrust 

55 

70 

control  actuation,  psia 

• 

Thrust  chamber  metal  temperatura,  °R 

300 

570 

Thrust  Control 

Time  Constant 

0.55  0.30 

Thrust 

570°R 

55  psia  |  70  pala 

Chambsr 

_ Thrust  Control  _____ 

Mstal 

Temperature 

300°R 

Body  Pressure 

70  psia  55  pala 

Note  that  this  program  required  only  four  run*,  while  a  Full  Factorial 
would  have  required  eight  run*  (23). 

From  th«  reiult*  of  thi ■  program  it  wa»  determined  that  both  thrust 
■octroi  constant  and  body  praiiure  had  Important  main  cffcctc  in  deter¬ 
mining  thrust  overshoot.  The  role  of  thrust  chamber  metal  temperature 
in  determining  overshoot  verified, 

Subsequent  to  this  program,  development  was  continued  on  the  thrt'si 
.-.ontrol  t<->  limit  thrust  overshoot.  This  development  uffort  required  the 
running  of  several  Full  Factorial  programs  along  tha  lines  suggested  by 
the  results  of  the  original  Latin  Square  program. 

C.  THE  RANDOM  BALANCE  TEST 

If  a  largs  number  of  factors  sre  believed  to  be  Influential  In  obtaining 
a  certain  result,  a  simple  Full  Factorial  may  Involve  an  excessive  number 
of  tost*.  For  example,  if  there  are  10  factors  each  at  three  levels,  a 
Full  Factorial  would  require  a  total  of  59,049  tests. 
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A  random  representative  sample  of  euch  a  Full  Factorial  u  the 
Random  Balance -designed  experiment  can  be  used  to  reveal  the  stronger 
main  effects  and  interactions  of  the  10  variables.  The  combinations  of 
levels  of  each  variable  for  sach  test  are  then  chosen  at  random.  If  30 
tests  were  permitted  in  this  design,  then  each  of  tho  5  levels  of  each 
factor  would  be  tested  10  times.  A  Random  Balance  design  may  be 
anaiyaed  with  McBee  edge  punch  cards,  graphical  regression  analysis, 
and  tests  of  significance,  including  analysis  of  variance. 

If  there  are  suspected  interactions  between  several  factors,  a 
Multiple  Balance -designed  experiment  would,  with  almost  no  loss,  more 
completely  evaluate  such  suspected  interactions.  For  example,  if  3  of 
the  10  factors  are  expected  to  be  interacting,  the  3  variables  are  laid  out 
in  a  Full  Factorial,  33«27  tests,  with  three  of  these  cells  (chosen  at 
random)  replicated  or  repeated  for  a  total  of  30  tests.  The  tests  of  this 
factorial  are  thun  listed  In  the  random  order  of  running  and  the  remaining 
•even  variables  have  their  levels  randomly  but  aqually  distributed  through¬ 
out  these  30  runs.  The  analyses  are  the  same  with  the  exception  of  the 
analysis  of  the  factorial.  This  factorial  analysis  positively  identifies 
significant  interactions.  Although  in  theory  a  great  deal  of  information 
about  the  interactions  of  all  of  the  parameters  is  lost,  most  problems 
yield  to  this  test  approach  in  practice  because  the  one  or  two  most  im- 
pnnant  factors  or  combinations  of  factors  are  separated  from  the  unim- 
p.'fUnt  majority.  This  assumes,  of  course,  that  the  factors  considered 
for  the  random  or  multiple  balance  experiment  were  «sl»cl«d  with  good 
engineering  judgment, 

l>irlng  tho  course  o*'  development  of  the  RL  10,  It  became  apparent 
that  tbruut  conn*"’,  hysteresis  or  nonlinearity  was  a  problem.  This  was 
«vid«'ivtsd  by  failure  of  tho  thrust  control  to  repeat,  for  sucr.ossivn  runs, 
a  glv-n  setting  of  thrust  and  mixture  ratio  during  steady-etate  operation. 

By  applying  engineering  judgment  the  number  of  suspected  cause*  wh« 
reduced  to  10  factors,  which  are  listed  below,  Factors)  A,  R,  C,  and  D 
were  suspected  as  being  most  important,  while  factors  A,  B,  and  C  were 
suspected  to  Interact.  The  number  in  parenthesis  after  each  of  the 
lectors  indicates  the  levels  of  each  of  the  factor*. 

A  Bellow*  Assemblies  (4) 

D  Reference  Spring  (2) 

C  Ball  or  Race  Tuldisa  (2) 

D  Bypass  Valvo  Assembly  (2) 

E  Bypass  Valve  Spring  (2) 
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l1'  Lower  Housing  (2) 

G  Feedback  Spring#  (2) 

H  Deference  Spring  Guide#  (2) 

1  Carriage  (bellows)  (2) 

J  Method  of  Thrust  Control  Assembly  (2) 

Q 

A  Full  Factorial  of  the  10  factors  would  require  3072  tests  (4x3x2  ) 
to  obtain  complete  knowledge  of  all  interactions.  Since  factors  A,B,  and 
C  were  suspected  to  Interact,  a  Full  Factorial  of  these  factor#  wa#  built 
into  a  Random  Balance  experiment,  thus  making  it  a  Multiple  Balance 
experiment.  It  was  pointed  out  that  analysis  could  begin  on  about  the 
tenth  teat. 

The  32  tests  are  shown  as  follows,  It  will  be  noted  that  the  Full 
Factorial  of  factors  A,  B,  and  C,  with  two  replicate#  per  cell,  were 
actually  run  with  tho  remaining  factors  selected  at  random,  thus  a 
Multiple  Balance  design,  as  shown  on  the  next  page. 
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Test  No. 

A 

B 

c 

D 

E 

F 

G 

H 

J 

j 
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3 

3 

- 

+ 

4 

4- 
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2 

.1 

4 

4 

+ 

+ 

- 

- 

2 

+ 

4 

5 

4 

- 

* 

- 

- 

- 

2 

6 

i 

- 

4 

4- 

- 

- 

1 

4- 

„ 

V 

3 

+ 

4 

- 

4- 

- 

1 

„ 

4- 

8 

2 

- 

+ 

4- 

+ 

+ 

3 

9 

1 

4- 

- 

- 
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4 

3 

10 

2 

4 

- 

* 

4- 
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. 
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4 

+ 
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4 

+ 
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3 

4- 

12 

2 

- 

+ 

- 

- 

- 

+ 

1 
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4 

13 

1 

4 

• 

- 

- 

4- 

4 

2 

4- 

+ 

14 

4 

4 

+ 

4- 
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• 

2 
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+ 

IS 
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- 

4- 

• 

• 

. 

M 

2 

16 

3 

- 

- 

4- 

4- 
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.. 
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17 

2 
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The  results  of  these  tests  showed  that  Factors  I  (carriage),  J 
(assembly),  A  (bellows),  D  (bypass  valve  assembly) ,  and  F  (lower  housing) 
have  the  greatest  main  effects.  Interestingly,  an  interaction  between 
Factors  A,  B,  and  C  was  not  conclusively  demonstrated.  The  combination 
of  levels  of  the  factors  which  gave  minimum  thrust  control  hysteresis  was 
sufficiently  demonstrated  by  this  test  to  enable  a  satisfactory  solution 
of  the  hysteresis  problem  to  be  found.  Thus,  the  problem  was  solved  in 
a  32-run  program  in  a  far  shorter  time  than  would  have  been  possible 
with  the  conventional  cut-and-dry  testing  techniques, 

CONG LU SIGNS,  Great  advantages  are  possible  with  statistical 
testing  techniques  if  certain  guide  lines  arc  observed: 

1,  The  urge  to  believe  that  mid-program  results  have  solved  the 
problem  is  strong  and  must  be  resisted.  The  program  must  be  carried 
through  to  its  conclusion. 

2,  The  tests  must  be  run  in  a  random  fashion  even  though  this  is 
not  necessarily  the  fastest  approach.  For  example,  running  the  first 
half  of  a  program  exclusively  on  Stand  A,  and  the  second  half  of  a  pro¬ 
gram  on  Stand  B,  may  add  a  degree  of  oonfoundlng  that  would  not  have 
occurred  if  random  stand  changes  had  been  made.  If  such  tests  must 
be  run  In  a  nonrandom  fashion,  or  in  blocks,  the  results  must  be 
analysed  with  this  in  mind. 

3,  Statistical  testing  is  not  a  substitute  for  sound  engineering  Judg¬ 
ment  but  simply  a  method  for  obtaining  the  most  efficient  testing  program 
There  is  no  conflict  between  sound  engineering  judgment  and  statistical 
testing  techniques. 


COMPUTER  SIMUWtTION  STUDY  OF 
BRUCETON  AND  PROBIT  METHODS  OF  SENSITIVITY  TESTING 

John  B,  Gayle 

National  Aeronautics  and  Space  Administration 
George  C.  Marshall  Space  Flight  Center 
Huntsville>  Alabama 

SUMMARY.  A  computer  simulation  technique  has  been  used  to  evalu¬ 
ate  the  suitability  of  the  Bruceton  and  Ptobit  methods  of  sensitivity  testing 
for  launch  vehicle  applications.  The  results  indicated  that  either  method 
permits  satisfactory  estimates  for  the  mean  value  of  a  normal  distribution,  ] 
However,  for  launch  vehicle  applications,  estimates  of  the  levels  of  stimu¬ 
lus  corresponding  to  very  high  or  very  low  percentage  reactions  usually 
are  required)  neither  method  permits  reliable  eetimates  for  such  levels 
when  the  distribution  Is  non-normal,  a  condition  which  occurs  with 
sufficient  frequency  that  assumption  of  a  normal  distribution  cannot  be 
justified  as  routine  procedure. 

INTRODUCTION.  The  rapid  growth  of  a  launch  vehicle  technology  has 
resulted  in  the  widespread  application  of  a  variety  of  materials  which  are 
capable  of  reacting,  either  alone  or  with  aoma  other  material,  in  the  form 
□  f  an  explosion  when  subjected  to  a  suitable  stimulus.  Test  methods  for 
determining  the  extent  of  the  hasard  associated  with  any  particular  material 
or  material  combination  usually  consist  of  subjecting  samples  to  prede- 
‘ermined  stimuli  and  noting  the  frequency  of  reactions.  Because  of  the  j 

ni)-or-none  nature  of  the  results,  a  series  of  tests  usually  is  conducted  f 

Ir  v,hlch  replicate  samples  are  subjected  to  different  etlmull  and  the  re-  1 

suns  subjected  to  some  form  of  statistical  analysis,  the  details  of  vhlch 
may  be  dictated  by  the  pr.rtii.uiar  test  procoduro  employed.  The  method 
moot  generally  accepted  in  the  explosive  industry  is  the  Bruceton  up-and- 
down  method  which  specifies  both  the  experimental  procedure,  an d  the 
statistical  analysis.  An  equally  applicable  method  which  is  more  genet«l!" 
ao  '.rptcd  in  the  biological  sciences  is  the  Probit  method  which  also  spect-  ■ 

fieri  both  the  experimentsl  procedure  and  the  statistical  analysis.  I 

The  purpose  of  this  investigation  was  to  determine  the  suitability  of  ! 

eacn  of  these  methods  for  lsunch  vehicle  applications,  I 
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PROCEDURE,  The  amount  of  information  which  can  be  obtained 
from  any  laboratory  study  of  a  particular  material  usually  Is  limited  to 
that  derived  from  the  selected  test  method  and  no  ab»oluta  basis  exists 
for  determining  the  extent  to  which  this  Information  Is  actually  doscrlp- 
tive  of  the  material  being  tested.  Available  information  also  is  severely 
limited  by  the  number  of  tests  which  can  be  Included  In  any  experimental 
study,  This  limitation  Is  particularly  important  for  all-or-nune  type  data 
for  which  a  minimum  of  10  to  100  Individual  tests  are  required  to  obtain 
even  a  preliminary  result, 

To  circumvent  these  and  similar  difficulties!  the  approach  used 
In  this  Investigation  consisted  of  generating  data  for  "systhetlc  ex¬ 
plosives"  of  rigorously  defined  characteristics  and  comparing  the  various 
population  parameters  with  those  determined  by  the  Bruceton  and  Pro¬ 
bit  methods  of  analysis, 

A,  BRUCETON  OR  UP-AND-DOWN  METHOD 

This  method  has  been  described  by  several  investigators  (Ref.  1). 

In  practice,  the  range  of  stimuli  which  can  be  applied  in  a  given  test 
apparatus  Is  divided  into  discrete  Increments  of  uniform  spacing,  Thus, 
lor  an  Impact  apparatus  for  which  the  height  of  drop  can  be  varied  from 
n  to  60  Inches,  one  can  define  31  levels  separated  by  distances  of  2 
inches#, 

'  o  actually  carry  uut  a  test,  an  initial  level  le  selected  which  lu 
guessed  to  be  close  to  ihe  rtif.r.n  value  end  a  single  sample  tested  at 
that  level,  it  an  explosion  or  ether  reaction  Is  noted,  a  plus  le  re¬ 
corded!  if  not,  a  minus,  A  esc  end  sample  then  1*  tested  at  the  next 
lowc.t  level  if  a  plus  wee  recorded  for  the  previoue  sample,  or  at  tie 
next  higher  level  If  a  minus  was  recorded  for  the  previous  sample. 

A  till  rd  sample  then  is  tested  in  the  same  manner  and  eo  on  until  thu 
scheduled  scries  of  tests  le  completed. 


>1, 

For  some  applications,  It  Is  advisable  to  convert  sll  heights  to  log 
units  and  separate  the  levels  by  uniform  Increments  measured  In  log 
units.  However,  except  for  this  transformation  of  units,  the  test  Is 
carried  out  in  the  usual  manner. 
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F-'i-  purpose*  of  calculation,  either  the  pluses  or  minute*  may 
be  used)  however,  the  reiult*  for  the  firat  few  teit*  are  dlacarded  up 
to  but  not  including  the  reiult  ju*t  prior  to  the  tint  change  in  *lgn  (plu* 
to  minus  or  mlnu*  to  plus).  Further  detail*  of  the  calculation*  have 
been  adequately  described  and,  therefore,  are  not  Included, 

The  method  yield*  a  mean  value,  corresponding  to  the  level  at 
which  there  1*  a  50/50  probability  of  a  reaction,  a  atandard  deviation 
which  ia  used  to  compute  the  level  correaponding  to  any  other  proba¬ 
bility  of  a  reaction,  and  a  atandard  deviation  of  the  mean,  It  ahould 
be  noted  that  the  up-and-down  aapect  of  the  toet  procedure  tenda  to 
compress  the  daw  about  the  mean  value  and,  conaequently,  yields  a 
blaied  (low)  eetlmate  of  the  atandard  deviation  which  ie  subsequently 
adjuated  to  obtain  an  unbiased  eatimate, 

To  carry  out  a  computer  aampling  study,  synthetic  exploaivea 
were  defined  for  which  the  probability  of  a  reaction  correaponding  to 
any  given  level  could  be  determined  by  use  of  a  table  of  random  digit*. 
Consider,  for  example,  a  level  for  which  the  probability  of  a  reaction  ia 
required  to  be  70  percent,  A  simulated  teat  conslsti  simply  of  drawing 
a  pair  of  random  digit*  and  making  a  teat  to  aae  whethar  or  not  the 
numerical  value  of  the  pair  ia  equal  to  or  lee*  than  69,  If  ao,  a  plus 
in  recorded)  if  not,  a  minus.  The  level  for  the  next  teat  ia  selected 
by  adding  or  subtracting  one  from  the  previous  level  (depending  on  the 
outcome  of  the  previous  test)  and  the  aerie*  continued  in  the  name 
manner  a*  would  be  the  case. in  an  actual  experiment, 

A*  a  matter  of  convenience,  all  of  the  teat  data  we**  used  for 
computations,  none  of  the  iiilial  values  being  discarded,  However,  ell 
initial  teste  were  madu  at  It  v  vis  'or  which  the  probability  of  a  reaction 
««•  in  the  range  0  <  %  <  1 00 ,  and  the  effects  of  entering  the  progi-.n  t 
random  level*  and  at  different  fixed  level*  were  considered.  Each  aerie* 
of  teat*,  consisting  of  i 00  samples,  was  repeated  50  timer,  and  standard 
methods  of  statistical  analysis  were  used  to  determine  the  reproduci¬ 
bility  of  the  sample  estimates, 

The  ievels  corresponding  to  a  5  percent  probability  of  a  reaction 
were  estimated  from  the  averages  of  the  mesne  and  standard  devi¬ 
ations  for  each  group  of  50  series. 
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B.  PROSIT  METHOD 

The  experimental  portion  of  this  method  differs  from  the  Bruce- 
ton  method  principally  in  that  the  individual  testa  are  carried  out  at 
levels  of  stimuli  selected  by  the  investigator  rather  than  those  dic¬ 
tated  by  the  preceding  test  result  (Ref,  2)  .  Although  both  the  par¬ 
ticular  levels  selected  and  the  number  of  tests  made  at  each  level 
may  be  varied  within  wide  limits,  in  practice,  approximately  5  levels 
usually  are  selected  which  are  guessed  to  fall  close  to  and  on  either 
side  of  the  mean,  and  the  number  uf  tests  at  each  level  is  constant, 
usually  around  20, 

The  novel  feature  of  this  method  Is  the  transformation  of  the  non¬ 
linear  percentage  reaction  versus  stimulus  level  plots  obtained  directly 
from  the  data,  to  linear  plots  (assuming  a  normal  distribution)  which 
Is  accomplished  by  expressing  the  reaction  frequencies  in  standard 
deviation  units  referred  to  as  "probits,  "  Subsequent  manipulation  of 
the  transformed  data  yields  estimates  of  the  mean,  standard  deviation, 
and  standard  deviation  of  the  mean. 

The  computer  sampling  study  for  this  method  was  carried  out 
using  the  same  synthetic  explosives  or  populations  that  ware  used  with 
ti i '■  Bruceton  method,  the  levels  for  testing  being  varied  to  determine 
th"  extent  to  which  the  particular  levels  selected  Influenced  the  re¬ 
sults,  Again  100  samples  were  used  for  each  test  aeries,  20  being 
taken  at  each  of  5  levels,  Also,  each  test  sarles  was  ropeatsd  ». 
tot.it  of  SO  times  to  permit  direct  estimates  of  the  reproducibility  of 
dntu  obtained  by  this  method.  The  levels  corresponding  to  S  percent 
reaction  were  again  estimated  item  the  average  results  for  each  group 
of  SO  series, 

C.  RESULTS 

As  Indicate d  above,  each  scries  of  100  whether  made 

using  the  Bruceton  or  Problt  method  of  analysis,  provides  cellmates 
of  the  mean,  standard  deviation,  and  standard  deviation  of  the  met 
By  repeating  each  sertea  SO  times,  average  values  were  obtained  for 
each  of  these  estimates  based  on  the  calculations  peculiar  to  etch 
particular  method  of  testing,  In  addition,  application  of  conventional 
methods  of  atatlstical  analysis  to  each  group  of  SO  results  provided 
estimates  of  the  reproducibility  of  these  values.  Population  values 
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for  the  mean,  standard  deviation*,  and  standard  deviation*  of  the  mean 
were  calculated  from  the  frequency  distribution*  in  the  usual  manner; 
the  levels  corresponding  to  5  percent  reaction  were  interpolated  from 
plot*  of  tlie  cumulative  frequency  distribution*  for  each  population. 

The  result*  for  the  Bruceton  and  Problt  method*  of  analysis 
are  not  strlntly  comparable  because  of  the  influence  oi  the  point  of 
entry  for  the  Bruceton  method,  and  the  particular  level*  selected  for 
testing  for  the  Probit  method.  However,  general  comparison*,  such  aa 
those  given  in  the  following  sections,  are  believed  to  be  indicative  of 
the  relative  characteristics  of  the  two  methods, 

The  several  populations  or  "synthetic  explosives"  selected  for 
testing  are  shown  graphically  in  FIG  1,  Also,  the  probabilities  of  a 
reaction  at  any  given  level  are  eummarieed  for  each  population  in 
Table  1,  It  should  be  noted  that  the  extreme  ends  (1  to  2  percent)  of 
esch  distribution  were  truncated  for  convenience  in  eimulatlng  test 
data  by  using  pairs  of  random  digit*.  These  populations  include  both 
normal  and  non-normal  (peaked,  akewed,  and  bimodal)  types, 

Application  of  either  the  Bruceton  or  Problt  method  aasumes 
that  the  distribution  of  data  1*  normel.  Approximately  normal  distri¬ 
bution*  therefore  were  selected  for  the  first  two  populations  to  obtain 
data  under  Ideal  conditions  of  testing, 

Table  2  summarises  the  results  obtained  by  the  Druoeton  method 
for  population  No.  1.  Table  3  give*  s  Imilar  results  for  population  No,  ? 
v'.ich  differs  largely  in  that  the  50  percent  level  was  shifted  from  a 
population  value  of  t,  Cn,  wnich  coincides  with  one  of  the  levels  selected 
for  testing,  t"  1.  5  which  is  midway  between  two  of  the  levels  selected 
fur  testing, 

The  estimated  mean  values  are  in  excellent  agreement  wnh  the 
population  values.  Also,  the  estimated  standard  deviations  and  standard 
deviations  of  the  means  are  very  close  to  th*  population  values  with 
some  slight  bias  in  the  direction  of  low  estimate*  being  evident,.  TM* 
biu*  results  in  slightly  high  estimates  of  the  levels  corresponding  to 
5  percent  reactions,  but  the  discrepancy  is  of  doubtful  practical 
significance. 

Tables  4  and  5  summarise  the  results  obtained  by  the  Problt 
method  for  population*  No.  1  and  2.  When  the  level*  selected  lor 
teellng  arc  closely  grouped  around  the  mean,  all  of  the  estimates  are 
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Table  1,  '  Cumulative  Frequency  Distribution*  {or  Synthetic  Expletive* 


Value*  in  body  of  table  indicate 
reaction  frequency  at  given  leval,  cumulative  percent 


Population  Tvnc 

Normal 

Normal 

. Peaked . . 

Skewed 

Simodal 

Population  No, 

.3 

4 

3  . 

Level 

0 

0 

0 

0 

0 

u  0 

1 

2 

l 

4 

2 

3 

2 

9 

4 

5 

12 

8 

3 

23 

13 

.7 

30 

23 

4 

30 

37 

10 

30 

33 

3 

73 

63 

18 

68 

68 

6 

91 

83 

■  '  32 

82 

73 

7 

98 

96 

30 

90 

76 

8 

100 

99 

68 

93 

76 

9 

100 

82 

95 

78 

10 

90 

96 

83 

11 

93 

100 

93 

12 

93 

98 

'  13 

96 

100 

14 

100 

T 
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Tab  la  2.  Summary  of  Brueaton  Mathod  Ranult*  for  Normal  Llatrlbution 
(Population  No.  1) 


Laval  for 
First  Tost 

In  Sariaa 

Rasult 

led.  Brror 
of  Reault 

Moan  or  30%  laval 

Population  valua 

a* 

4.00 

m 

Sampia  aatlmata 

4 

3.99 

.03 

do 

£ 

4,05 

.03 

do 

2-6* 

4.01 

.03 

Standard  Davlatlon 
Population  valua 

. 

1.49 

•1 

Samp  la  aatlmata 

4 

1.37 

,04 

do 

6 

1.42 

.04 

do 

2-6* 

1.43 

.04 

Standard  Davlatlon  of 
Population  valua 

Ha  an 

M 

0.21 

• 

Sampla  aatlmata 

4 

0.19 

.00 

do 

6 

0.19 

.00 

do 

2-6* 

0.20 

.01 

3%  Laval 

Population  valua 

- 

1.50 

at 

Sanpla  aatlmata** 

4 

1.74 

• 

do 

6 

1.71 

aa 

do 

2-6* 

1.66 

m 

*  Tha  laval  for  tna  fine  cat t  In  eash  aarlaa  waa  aalaotad  at  random 
from  lava  la  2  through  6. 

**  <M«l .645  c ■  )  eompuCad  from  ovarall  avaraga  valuaa  for  n.tan  and 

atandard  davlatlon. 


T«bl«  3.  Summary  of  Brueeton  Mat  hod  Ranult*  for  Normal  Dletribution. 
(Population  No.  2) 


Laval  for  Dtd.  Error 

flrit  Taat  ReeuU  of  Reault 

■VP  Jgtiaa. — . . . . — — 


H««n  or  507.  level 
Population  value 
Samp  la  aatimata 

Standard  Oavlatlon 
Population  valua 
Sample  aatimata 


2-7* 


2-7* 


4.50 

4.49 


1.4B 

1.38 


0.03 


0.03 


Standard  Deviation  of  Mean 
Population  valua  • 

Sampla  aatimata  2 »7* 


0.21 

0.19 


0.00 


iX  level 

Population  value 
Sample  eaUmata** 


2-7 


2.08 

2.22 


*  The  level  for  the.firet  teat  in  each  earlea  wan  aalactad  at  random 
from  lava  la  2  through  7  except  that  the  proheoillty  of  selecting 
level*  2  or  7  wa«  one-half  chit  of  aeleetlng  3  through  6. 

**  <W-l.u45«'  t  computed  from  overall  average  vsluae  for  mean  and 

atandard  deviation. 


Z  1  5 


Table  4.  Summary  oi'  Probit  Method  Raaulta  for  Normal  Distribution 
(Population  No.  1) 


Lrvala  at 
Which  Xotts 
ware  made 

Result 

Std.  Error 
of  KaaulC 

Maan  or  50*4  level 
Population  value 

4.00 

at 

Sample  eatimate 

2-6 

4. no 

1 18 

do 

0-4 

2.09 

.04 

Standard  Deviation 

Popu lotion  value 

- 

1.49 

- 

Sample  eatimate 

2-6 

1.51 

.04 

do 

0-4 

1.86  > 

.12 

Standard  Daviation  of  Maan 

Population  valua 

m 

0.21 

Sample  tatimata 

2-6 

.22 

.00 

do 

0-4 

.32 

.02 

3%  laval 

Population  value 

• 

1.38 

* 

Sample  eatimate* 

2-6 

1.32 

aa 

do 

0-4 

-.17 

" 

*  (M-1.643  t  )  eomputad  from  ovarall  avaraga  valuta  for  mean 

and  standard  daviation. 
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Table  3.  Summary  of  Preble  Method  Raiulce  for  Normal  Distribution 
(Population  No.  2) 


Ltvale  at 
Which  Te»t» 
_ were  made 

Reaulta 

Std  Error 
of  Reeult 

Maan  or  30a  level 

Population  value 

■ 

4.30 

• 

Sample  eatlmata 

2-6 

_  4.24 

0.02 

do 

0-4 

3.08 

0.03 

do 

4-8 

3.21 

O.C 

Standard  Deviation 

Population  value 

- 

1.48 

» 

Sample  eatlmata 

2-6 

1.46 

0.04 

do 

0-4 

2.72 

0.43 

do 

4 -A 

1.44 

0.33 

Standard  Deviation  of  Maan 

Population  value 

■ 

0.21 

• 

Sample  eatlmata 

2-6 

0.22 

0.00 

do 

0-4 

0.32 

0,08 

do 

4-8 

0.23 

0.01 

37.  Level 

Population  value 

Sample  eatlmata* 

- 

2. OS 

• 

2-6 

1.64 

- 

do 

0-4 

*1.38 

m 

do 

4-8 

2.84 

m 

*  (H-1,643  9  )  somputed  from  ovarall  average  values  for  mean  and 
standard  deviation. 
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Table  6.  Summary  of  Bruceton  Method  Raeulta  forPoakad  Dleerlbutlon 
(Population  No.  3) 


Level  for 
Plrat  Teat 

In  Sorioa 

Result 

Std.  Error 
of  Result 

Mean  or  30%  level 
Population  value 

7.00 

Sample  eetimate 

3- 11* 

7.06 

0.04 

Standard  Deviation 

Population  value 

• 

.2.69 

.  ac 

Sample  aitiraat*  . 

3-11* 

2.21 

0.09 

Standard  Deviation  of  Mean 
Population  value  * 

0. 38 

Sample  eatimate 

3-11* 

0.29 

0.01 

3%  level 

Population  value 

- 

2.00 

• 

Sample  eatimate** 

3-11* 

3.42 

• 

*  The  level  for  the  flratt  teat  In  each  eerlae  wee  selected  at  random 
from  levels  3  through  11  except  that  the  probability  of  aelectlon 
of  level  7  wee  twice  that  for  the  other  levele. 

**  (M-1.645v)  computed  from  overall  average  value*  for  mean. and 
atanderd  deviation. 
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Table  1.  Bummaiy  o f  Bruseton  Method  Remit*  for  Skewed  Diatrlbutlon 
(Population  No.  4) 


level  for 
Firet  Teat 
_ in  Sat  lea _ 

Reault 

Std.  error 
of  Roeult 

Mean  or  30%  level 

Population  value 

- 

4.32 

- 

Sample  aatlmat* 

1-10* 

4.21 

0.04 

Standard  Deviation 

Population  value 

2.20 

m 

Semple  aatlmat* 

1-10* 

2.33 

0.12 

Standard  Deviation  of  Maan 
Population  val.ua 

• 

0.31 

. 

Sample  eatimat* 

1-10* 

0.31 

0.01 

3%  Level 

Population  value 

a* 

1.45 

a» 

8ample  eatimata** 

1-10* 

0.34 

• 

*  The  level  for  the  firm  tent  in  eeoh  eerie*  wee  aalectad  at  random 
from  level!  1  through  10. 


**  (H-l .  443  <r  )  computed  from  overall,  aver*!*  value*  for  mean  and 
,  *  tender d  deviation. 
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Table  8  Dwmtaty  of  Treble  Machod  Raiultt  for  Peeked  file eribuelon 
(Population  Ho,  3) 


Lavala  at 

Which  Teata 
vara  made _ _ 

Result 

Std.  Error 
of  Raault 

M«an  or  50ft  Ltvel 

Population  value 

*  m 

7.00 

• 

Sample  baeimata 

5-3 

7.01 

0.01 

do 

0,3,6,9,12 

6.92 

0.06 

do 

0-4 

2.63 

0.08 

Standard  Deviation 
Population  valua 

. 

2.69 

m 

Sample  aaeimata 

5  -9 

2.18 

0.07 

do 

0,3,6,9,12 

0-4 

6.37 

2.73 

do 

6.14 

1.13 

Standard  Deviation  of  Mean 

Population  valua 

- 

0.38 

as 

Sample  aatlmata 

3-9 

0.30 

0.01 

do 

0,3,6,9,12 

1.39 

0.62 

do 

0-4 

1.30 

0.27 

n  Laval 

Population  valua 

Sample  aatlmata'* 

- 

2. 00 

• 

3-9 

3.42 

«e 

do 

0,3,6,9,12 

-3.89 

at 

do 

0-4 

-7.23 

* 

*  (M-l . 64J  ir  )  remput.d  from  overall  average  valuai  for  naan  and' 
atandard  deviation. 
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Table  y,  Summary  oi  Probit  M* thod  RuuUt  for  Skewed  Dlatributlon 
{Population  No.  4) 


Lovala  at  Which 

Taata  ware  mada 

Result 

ltd.  Error 
of  Boault _ 

Moan  or  SOX  Laval 

Population  valua 

4.32 

aa 

Samp la  aatimata 

3-7 

4,72 

0.02 

do 

0,2. 4, 6, 8 

4.27 

0.04 

do 

0-4 

.  2.82 

0.03 

standard  Deviation 

Population  valua 

m 

2.20 

« 

Sample  aatimata 

3-7 

2.22 

0.07 

do 

0,2.4, 6,8 

7.00 

4.10 

do 

0-4 

1.61 

0.08 

Standard  Daviatlon  of  Maan 

Population  valua 

•a 

0.31 

m 

lampla  aatimata 

3-7 

0.31 

0.01 

do 

0,2, 4.6.8 

0-4 

1.49 

0.93 

do 

0.3 

0.01 

3%  Laval 

Population  valua 
lampla  aatimata* 

• 

1 .43 

■ 

3-7 

1.07 

4 

do 

0,2, 4, 6, 8 

-7.23 

m 

do 

0-4 

-0,16 

* 

*  <M- 1.645  9  )  commuted  frcii*  ovoroli  ivirift  valuaa  for  swan  end 
atandurd  deviation. 
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labia  10,  Summary  of  Brucaton  Mathod  Raiulta  for  Bimodal  Distribution 
(Population  No.  5) 


Lavol  for 
firat  Taat' 
in  Sarlaa 

ftaault. 

8td.  Error 
of  Raaulta 

Maan  or  50%  Laval 

Population  valua 

5.18 

aa 

Sampla  aatlmata 

2 

4.22 

0.05 

7 

4  .'40 

0.04 

do 

U  . 

4.84 

0.04 

do 

2-11* 

4.47 

0.05 

Standard  Daviation 

Population  valua 

- 

3.23 

- 

Sampla  aatlmata 

2 

2.10 

0.10 

do 

7 

2.53 

0.15 

do 

11  ^ 

5.41 

0.17 

do 

2-11* 

3.24 

0.23 

Standard  Daviation  of  Maan 
Population  valua 

to 

0.46 

. 

Sampla  aatlmata 

2 

0.28 

0.01 

do 

7 

0,32 

0.02 

do 

11  - 

0,44 

0.06 

do 

2-X1* 

0.33 

0.06 

St  Laval 

Population  valua 

Sampla  aatlmata*'" 

• 

1.60 

2 

0,77 

do 

7 

0.24 

m 

do 

11 

-4.06 

at 

do 

2-13* 

0.86 

aa 

*<  Tli a  lavol  for  tha  flrat  Ut.  waa  aalaotad  at  random  from  lsvala  2 
through  11, 

**  (M-1,645  <r  )  computad  from  ovarall  avaragt  valuta  for  maan  and 
lUndard  daviation, 


222 


Design  of  Experiments 


in  excellent  agreement  with  the  population  values,  However,  when  the 
levels  selected  for  testing  are  distributed  solely  below  the  mean,  the 
deviations  between  the  sample  estimates  and  population  values  for  the 
mean  and  standard  deviations  become  appreciable,  and  the  sample  esti¬ 
mates  of  the  levels  corresponding  to  5  percent  probability  of  reactions 
deviate  markedly  from  the  population  values, 

Tables  6  through  9  summarise  results  for  two  populations  which 
are  either  peaked  or  skewed  and  thus  depart  appreciably  from  normality 
as  shown  in  FIG  1,  For  the  Bruceton  method,  the  mean  and  standard 
deviations  for  population  No.  3  are  in  good  agreement  with  the  popu¬ 
lation  values,  whereas  the  deviation  for  the  level  corresponding  to  5 
percent  reactions  is  somewhat  larger.  This  discrepancy  is  caused  by  the 
decreasing  slope  of  the  cumulative  distribution  curve  shown  in  FIG  1  for 
values  below  approximately  15  percent,  For  population  No,  4,  the  slope  of 
the  distribution  curve  increases  for  this  same  range  of  values,  and  the 
agreement  between  nnnulatlon  values  and  sample  estimates  for  the  5 
percent  level  Is  slightly  better. 

Frobit  method  results  for  populat.on  No.  3  are  similar  to  those 
for  the  Bruceton  method  when  the  levels  selected  for  testing  are  closely 
grouped  around  the  mean.  However,  when  the  Levels  were  dispersed  over 
the  entire  range  of  thole  available,  the  agreement  deteriorated  appre¬ 
ciably.  When  the  levels  were  limited  to  the  lower  half  of  the  distribution, 
the  sample  estimate  of  the  level  corresponding  to  5  percent  reactions  was 
markedly  In  error  in  the  opposite  direction  to  that  noted  when  the  samples 
were  closely  grouped  around  the  mean.  Results  for  population  No,  4 
exhibited  somewhat  different  uc-ds  with  the  5  percent  levels  estimated 
from  samples  closely  grouped  around  the  moan  end  those  limited  to  the 
lower  half  of  the  distribution  belns  in  closer  agreement  with  the  popu¬ 
lation  alues,  than  that  for  which  the  sampling  levels  were  dispers.J 
over  tne  entire  range. 

Tables  10  and  11  summarize  results  for  a  bimodal  population,  No.  5  , 
(i.  o,  ,  the  distribution  resulting  from  combination  two  uiher  distri¬ 
bution'/)  such  as  might  be  expected  for  the  mixtures  of  diverse  chemi'?' 
compounds  used  for  proprietary  materials.  Bruceton  method 
estimates  of  the  5  percent  levels  were  particularly  sensitive  to  the 
Level  selected  for  the  first  test.  Inspection  of  some  of  the  individual 
test  data  indicated  that  when  the  first  test  of  a  series  was  made  in  that 
portion  of  the  combined  distribution  corresponding  to  the  larger  of 
the  two  single  distributions,  there  was  ar  appreciable  number  of  in¬ 
stances  in  which  the  entire  series  was  confined  to  that  portion  of  the 
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distribution.  For  these  eer'js,  the  characteristic s  of  the  smaller  dis¬ 
tribution  would  have  little  influence  on  the  results.  This  situation) 
did  not  occur  when  the  first  level  selected  for  testing  was  in  that 
portion  of  the  combined  distribution  corresponding  to  the  smaller  of 
the  two  single  distributions.  However,  regardless  of  the  point  of  entry 
selected,  the  sample  estimates  of  the  5  percent  levels  were  not  con¬ 
sidered  tc  be  in  satisfactory  agreement  with  the  population  values, 

Probit  method  results  for  this  distribution  also  were  extremely 
sensitive  to  the  particular  levels  selected  for  testing  with  agreement  . 
ranging  from  poor,  in  some  instances,  to  ridiculous  in  others. 

DISCUSSION  AND  CONCLUSIONS.  The  re  suits;  reported  herein,  > 
ss  well  as  eimilar  results  for  several  other  populations  whlchhays 
not  been  included,  demonstrate  conclusively  that  tests  at  levels' lo- 
cated  close  to  Ihe  mean  value  of  a  normal  distribution  provide  excel¬ 
lent  estimates  of  the  characteristics  of  the  population  regardless  of 
whether  the  Brucston  or  Probit  method  is  used,  However,  when  the 
distribution  is  non-normal,  and,  particularly,  when  it  is  bimodal  in 
character,  the  estimates  provide  only  rough  indications  of  the  popu¬ 
lation  parameters,  and  in  particular,  the  estimates  of  tha  levels  '' 
corresponding  to  very  large  or  very  small  percentage  reactions  be¬ 
come  extremely  unreliable.  These  discrepancies  arc  the  direct  re¬ 
sult  of  linear  extrapolations  of  non-linear  data.  No  relief  from  this 
problem  is  possible  with  the  Bruceton  method,  which  is  specifically 
intended  to  concentrate  the  testing  at  levels  close  to  the  moan.  How¬ 
ever,  with  the  Probit  method,  the  operator  is  free  to  select  the  levels 
at  which  tests  s—  u.  iled  out  end  can  thus  concentrate  hie  efforts  on 
the  particular  end  of  the  distribution  of  greatest  interest  for  his  appli¬ 
cation,  The  results  obtained  tn  this  investigation,  however,  not  In¬ 
dicate  that  even  this  modification  provides  reliable  estimates  of  any 
of  the  statistical  parameters  considered  since  linear  extrapolation 
and  interpolation  are  atUl  used  with  non-linear  data. 

Most  launch  vehicle  applications  ere  concerned  with  either  Hit 
high  or  low  end  of  a  distribution.  In  fact,  probably  no  point  of  the  dis¬ 
tribution  is  of  leu  practical  significance  than  the  50  percent  point.  Thus, 
an  explosive  of  50  percent  response  to  s  given  stimulus  would  be  un¬ 
acceptable  from  either  a  performance  or  safety  point  of  view.  Con¬ 
versely,  levels  corresponding  to  very  large  or  very  small  percentage 
reactions,  usually  95 - 5  or  99- 1 ,  must  be  considered.  Results  given  in 
this  report  indicate  that  neither  the  Bruceton  or  Probit  methods  provide 
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reliable  estimates  of  itimull  corresponding  to  very  large  or  email  prob¬ 
abilities  of  reaction  unleei  the  population  is  normally  distributed,  For 
many  applications,  the  requirement  of  a  "normal  distribution"  may  be 
satisfied  if  only  that  portion  near  the  mean  approximates  a  normal  dis¬ 
tribution,  However,  for  reliability  and  safety  applications,  the  normality 
of  the  distribution  must  include  not  only  that  area  around  the  mean  but 
also  the  area  between  the  mean  and  the  percentage  frequency  of  interest. 
Assumption  of  such  normality  does  not  appear  justifiable  In  view  of  the 
heterogeneous  character  of  the  system*  undergoing  test.  Determination 
of  the  normality  or  lack  of  normality  of  such  a  system  would  require  ex¬ 
penditures  of  time  and  effort  in  excess  of  those  which  can  be  justified. 

It  is  recognised,  however,  that  valid  comparison*  of  parameters  for  re-' 
lated  distributions  are  sometime*  possible  even  when  the  aasumption  of 
normality  is  appreciably  In  error. 

To  obtain  additional  information  regarding  the  normality  of  distri¬ 
bution!  encountered  In  actual  testing,  results  of  JLOX  Impact  teati  on  a 
number  of  materials  were  examined,  Only  a  few  of  these  materials 
gave  results  which  could  be  reasonably  interpreted  as  normal,  the  dis¬ 
tributions  for  the  other  materials  varying  widely,  Fid  2  presents  typi¬ 
cal  curves  for  one  normally  (titanium  alloy)  and  two  non -normally  dis¬ 
tributed  sets  of  daU,  The  curve  for  pickled  titanium  is  of  particular 
interest  in  that  the  slope  of  the  curve  changes  sign  twice  within  a  rela¬ 
tively  narrow  range  of  impact  energies,  Ordinarily,  cumulative  fre¬ 
quency  distributions  are  obtained  by  cumulating  frequency  of  occurrence 
dot*.  The  nature  of  the  cumulating  process  is  such  that  changes  in  the 
s'jj-.i  of  the  slope  of  the  data  are  precluded,  Conversely,  for  sensitivity 
*<»-,  data,  each  point  on  the  curve  is  determined  directly  and  no  cumu¬ 
lating  operation  is  Involved.  The  changes  in  sign  noted  tor  these  data,  a 
and  also  for  data  reported  hv  other  invustigatore,  suggest  that  the  mech¬ 
anism  of  the  process  is  complex  and  varies  with  the  level  of  cumulus 
supplied.  It,  therefore,  is  debatable  as  Lu  whether  the  data  should  be 
ci  noidered  to  represent  a  frequency  distribution  in  any  sense  cf  the  word, 

In  view  of  these  considerations,  it  appears  that  the  best  method 
available  at  this  time  is  to  carry  out  a  systematic  search  for  a  eeni'Hvlty 
threshold  corresponding  to  some  acceptable  frequency  of  reaction  and  to 
utilize  graphical  and  statistical  techniques  to  evaluate  the  results.,  Such 
a  procedure  is  used  in  LOX  impact  testing  at  this  Center,  It  should  be 
noted  that  data  for  such  a  procedure  can  be  subjected  to  the  Probit 
calculations,  in  the  event  that  estimates  of  the  mean  and/or  standard 
deviation*  ate  considered  essential. 
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ABSTRACT,  Experiments  often  muBt  be  carried  out  with  less  than  (deal 
designs  because  of  limitations  having  their  origin  at  administrative  levels. 
Designs  may  be  modified  to  reflect  administrative  decisions  on  the  utililr.atlon 
of  available  resources  of  personnel,  funds,  and  equipment  in  a  research  pro¬ 
gram.  The  nature  and  effect  of  some  of  these  modifications  is  discussed,  An 
example  is  presented  and  some  alternatives  are  explored. 


Experimental  designs  often  are  influenced  by  administrative  limitations 
placed  upon  the  experimenter  or  the  research  unit.  The  result  may  be  a  frustrated 
researcher,  or  at  least  one  whoso  Ingenuity  may  be  sorely  tried  as  he  searches 
for  a  feasible  compromise  between  what  he  considers  an  ideal  design  for  ex¬ 
ploring  the  problem  and  the  practical  realities  of  the  situation.  This  may  also 
place  a  heavy  burden  on  the  statlstlclan-advisor  In  suggesting  a  statistical 
design  whlah  will  maximize  the  quantity  and  quality  of  information  obtained 
".ndor  somewhat  less  than  ideal  conditions, 

The  limitations  referred  to  are  those  having  their  origin  at  administra¬ 
tive  levels  rather  than  at  tne  research  level,  They  may  be  in  the  nature  of 
s.'sclflc  directives  related  to  particular  aotlvltlos,  merely  general  statements 
of  volley,  or  even  not  specifically  spelled  out  at  all.  The  Imposition  of  time 
limits  may  ho  nn*  f«*m  of  limitation.  The  necessity  for  an  early  administrative 
decision  may  l,;ud  to  a  request  for  a  quick  answer  to  some  research  problem. 
Perhaps  more  often  the  availability  ol  funds  or  personnel  is  aL  luce.  Closely 
related  is  the  question  of  priorities  for  various  projects.  Or  even  the  faoiibtns 
available  are  limited  and  expansion  is  impossible.  In  other  wcids,  we  art- 
concerned  with  the  impact  on  the  design  of  research  projects  of  administrative 
derisions  on  the  utilization  of  available  resources  of  personnel,  equipment 
and  funds  in  accomplishing  the  mission  of  the  organization. 

The  thought  might  be  Interpolated  here  that  the  experimenter  hlmsolf 
also  makes  many  similar  decisions  when  he  decides  how  he  is  to  divide  his 
time  and  effort  among  various  projects  or  segments  of  projects.  Actually  In 
practice  there  is  a  merging  ot  these  influences  in  the  final  decisions  regarding 

operation* 
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In  his  first  thinking  about  a  problem  an  experimenter  may  feel  that 
am  extensive  experiment  is  required  because  existing  knowledge  about  tire 
problem  Is  very  limited.  A  te- statement  of  the  problem  may  provide  a  setting 
for  developing,  within  the  various  limitations  present,  a  design  which  is  both 
feasible  and  statistically  valid.  It  is.  of  course,  always  assumed  that  rele- 
vant  findings  of  others  have  been  examined  to  provide  guidance  in  delimiting 
the  scope  of  the  project.  Repetition  of  the  work  of  others  simply  to  see  if 
you  can  duplicate  their  procedures  oan  be  a  wasteful  process.  This  is  not  to 
say  that  thura  may  not  at  times  be  reasons  to  repeat  eaperiments  of  others  to 
see  If  the  conditions  can  be  reproduced  and  confirmatory  results  obtained. 

But,  repetition  simply  for  the  sake  of  repetition  oen  be  wasteful  of  resources. 

Let  us  then  explore  some  of  the  consequences  of  the  Imposition  of 
administrative  limitations  on  the  conduct  of  experiments.  Adjusting  the  design 
to  meat  the  situation  can  be  done  in  many  different  ways.  It  has  already  been 
mentioned  that  a  restatement  of  the  problem  may  provide  the  basis  for  modifying 
the  original  proposal  Some  obvious  ohangss  quickly  come  to  mind.  A  smaller 
number  of  subjects  might  bo  used  or  tha  number  of  treatment  levels  reduoed. 

The  experimental  period  might  be  shortened.  Test  parameters  whloh  ere  to  be 
evaluated  might  be  restricted.  Sometimes  a  pilot  study  Is  an  economical  way 
to  establish  limits  within  which  the  final  study  Is  run,  With  careful  planning 
a  pilot  study  can  be  part  of  tha  Initial  stages  of  a  more  comprehensive  study. 
With  limits  established  for  the  area  of  primary  Interest  il  may  become  possible 
to  use  e  fairly  simple  design  with  straight,  forward  comparisons.  This  statement 
Is  not  Intended  to  Imply  that  It  may  not  he  desirable  to  uss  factorials  or  other 
des lu'.i  forms  which  Increase  the  information  available  more  rapidly  than  the 
cost. 


In  making  adjustments  in  the  design,  rare  must  be  exercised  to  Insure 
that  *h«  design  does  not  end  up  "unbalanced"  end  introduce  complice*' ons 
Into  the  statistical  analysis  and  the  Interpretation  of  the  results.  Unbalonau 
Is  noi  necessarily  fetal,  but  it  could  be  if  the  implloetions  to  the  analysis 
were  not  fully  anticipated  in  redesigning  the  experiment.  If  not  anticipated, 
the  statistician  might  bo  faced  with  e  "salvage  Job"  or  a  great  loss  of  informa¬ 
tion  bnoauso  an  important  part  of  the  data  became  unanalysable. 

How  will  the  statistician  meet  these  problems?  Mention  has  been 
made  of  the  use  of  smaller  numbers  of  subjects.  Standard  statistics  text 
books  contain  discussions  of  the  effect  upon  the  Inferences  to  be  drawn  of 
changes  In  the  numbers  of  subjects  or  observations.  Somewhere  along  the 
line  it  is  always  pointed  out  that,  th,-  uslBCtlon  of  the  number  of  observations 
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to  be  used  depends  In  part  on  the  variability  expected  In  the  various  parameters, 
and  on  the  precision  of  the  measurements.  The  probability  that  Inferences 
about  a  population  or  uni  versa  drawn  from  sample  observations  are  valid  de¬ 
pends  upon  such  factors  as  these.  Important  in  the  problem  then  Is  the  pro¬ 
bability  level  that  Is  considered  appropriate  in  the  case  of  «  particular  experi¬ 
ment. 


Except  In  the  most  unusual  situations  replicate  determinations  should 
be  performed.  Usually  duplicated  are  enough,  though  If  the  particular  proce¬ 
dure  Is  known  to  lack  precision  It  might  bB  well  to  uso  triplicates.  The  de¬ 
cision  lasts  on  the  variability  expected  to  ooour  In  the  Individual  measurements. 
The  variability  expected  among  the  subjects  In  turn  has  much  to  do  with  the 
number  of  subjects  to  be  used.  Equally  Important  here  Is  the  size  of  the  dif¬ 
ferences  between  treatment  groups  which  Will  be  considered  Important.  The 
sample  size  must  be  large  enough  to  detect  differences  that  are  important  In 
the  light  of  what  1s  known  or  estimated  about  the  variation  In  the  population. 

It  could  lead  to  unfortunate  consequences  If  decisions  on  these  matters,  fur 
example,  rested  solely  on  the  availability  of  personnel,  rather  than  taking  Into 
account  variations  of  the  types  just  mentioned.  However,  to  the  extent  that 
the  presence  of  administrative  limitations  forces  harder  thinking  and  more 
careful  planning,  and  thus  leads  to  a  "tighter"  design,  the  results  may  even 
be  beneficial  In  the  long  run. 

I  would  now  like  to  explore  with  you  a  specific  project,  the  design  of 
which  required  taking  Into  account  some  administrative  limitations  of  the  types 
“  have  been  talking  about,  Briefly,  the  purpose  of  this  project  is  to  toil  in 
the  field  five  different  rations  designed  for  non-resunpl',  situations  of  perhaps 
It1  days  duration  for  small  groups  of  n,°n.  atudy  parameters  include  certain 
biochemical  procedures  on  blood  and  urine  samples,  performance  tests,  and 
'ire  subjacts'  evaluation  of  the  rations,  Out  of  this  study  will  comu  at-'ements 
regarding  the  nutritional  adequacy  and  acceptability  of  the  tent  rations.  Also, 
ti  is  expected  that  suggestions  will  come  up  that  mignt  atd  In  the  development 
of  improved  rations  for  use  In  this  particular  type  of  field  situation. 

The  Ideal  da3ign  that  Immediately  comes  to  mind  Is  to  organise  5  patrol 
groups  and  set  up  a  5x5  lattn  square  pattern  for  feeding  the  rations.  In  this 
way  each  man  would  be  on  each  ration  sometime  during  the  test,  and  during 
each  cycle  each  ration  would  be  tested  under  the  same  prevailing  environmental 
conditions , 


Design  of  Experiments 


232 


The  time  required  would  be  the  10  days  In  the  field  during  each  cycle 
plus  a  rest  period  between  patrols.  A  rest  period  of  this  kind  serves  two  pur¬ 
poses.  One  is  a  recovery  from  the  prior  test  conditions  before  being  subjected 
to  a  new  test.  The  other  purpose  Is  to  give  time  for  remeasurement  of  basal 
study  parameters.  This  would  permit  reestablishment  of  normal  values  for 
each  man  Immediately  prior  to  entering  a  new  phase  of  the  experiment.  Inci¬ 
dentally  a  morale  factor  Is  also  Involved.  It  would  be  difficult  to  maintain 
morale  among  the  troops  If  they  were  on  patrol  continuously  for  a  period  as  long 
as  required  by  a  project  of  this  kind. 

It  was  pointed  out  that  a  full  schedule  on  a  5x5  latln  square  pattern 
would  tie  up  the  troops  and  test  personnel  Involved  for  nearly  90  days.  This 
was  considered  excessive  and  a  limit  of  approximately  55  days  was  set.  In 
partial  mitigation  It  was  determined  that  60  men  might  be  counted  on,  so  that 
6  test  groups  of  10  men  each  could  be  formed.  With  regard  to  the  test  rations 
it  was  noted  that  numbers  1,3,  and  5  were  oonsldordd  slightly  more  important 
than  2  and  4 . 

With  these  specifications  before  us  the  problem  was  to  Bet  up  a  work¬ 
able  design  for  the  field  work.  The  plan  established  provided  for  6  groups  of 
10  men  each  who  were  to  be  sent  on  10  day  patrols,  three  different  times.  The 
ration  lo  be  carried  and  consumed  by  each  group  was  assigned  by  a  random 
procedure  for  each  phase,  with  the  modification  that  rations  1,  3,  and  5  appear 
four  times  and  2  and  4  appear  three  times  (Table  1),  After  the  patrol  groupi 
are  formed  they  will  be  assigned  to  feeding  aequenoes  A  through  F  by  drawing 
out  Ur  a  hat  or  other  random  piucess. 

table  i  .  assignment  of  rations  to  groups  and  phases 

_ Patrol  group _ 

A  B  2  J?  1  Z. 

Phase  1  312543 

Phase  2  545231 

Phase  3  131425 


* 
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How  have  we  fared  In  batting  up  our  design?  We  have  assured  that 
no  patrol  group  tests  the  same  ration  twloe.  During  eaoh  phase  all  rations 
are  tested,  with  3,  S  and  1  eaoh  duplicated  In  one  of  the  three  phases.  The 
overall  time,  Including  preliminary  briefing,  "hefore"  and  "after"  studies, 

10  day  patrols,  and  5  day  rest  periods,  will  be  kopt  to  S5  days,  In  .terms  of 
the  administrative  limitations  Imposed  we  have  met  the  specifications. 

What  about  the  analysis  and  the  Inferences  to  be  drawn  from  the  results? 
For  any  of  the  tost  parameters  where  large  differences  appear  there  should  be 
no  problem.  It  Is  In  the  area  of  the  more  subtle  differences  that  thera  may  be 
a  problem .  Will  we  have  to  say  that  we  expect  considerable  variation  among 
the  men  because  of  the  neture  of  the  test?  Any  questions  dealing  with  the 
subjects1  evaluation  of  the  rations  will  have  ail  the  eubjeotlve  element*  that 
ouuui  in  all  food  acceptability  tests.  Tho  biochemical  procedures  art  reason¬ 
ably  prscise  and  o,  Jeotlve,  though  variation  among  normal  subjects  often  is 
great.  The  performance  teats  also  are  essentially  objective,  but  in  this  area 
there  always  Is  soma  question  sbout  the  effect  of  motivation  on  test  scores 
and  about  the  physiological  validity  of  the  tests  as  measures  of  response  to 
the  various  rations. 

|  in  this  test  our  first  Interest  is  In  the  nutritional  adequacy  of  the  diet 

ir,  maintaining  a  eoldlar,  oparattng  under  the  prescribed  conditions,  es  an 
efficient  fighting  maohtna.  Next  in  Importance  Is  the  acceptability  of  the 
ration  because  of  the  relation  of  acceptability  to  adequate  Intake  and  tc  morale 
generally. 

In  conclusion  then,  T  .suggest  that  in  a  test  of  the  hind  described, 
tne  design  should  oe  developed  es  far  es  possible  in  accord  with  sound 
statistical  practices,  but  tha.t  some  deviations  from  the  Ideal  are  not  neceti- 
e«niy  iatal  since  the  interest  Is  more  In  gross  differences  then  In  thwS"  of 
a  more  subtle  nature. 
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VERIFICATION  OF  PRODUCT  ACCEPTANCE 
INSPECTION  BY  ATTRIBUTES* 

Joeeph  Mandeleon 

Directorate  for  Quality  A aeurance,  Chamical  Corps  Material  Command 
Edgewood  Arsenal,  Maryland 

The  framework  of  reference  of  this  paper  is  tnat  of  Department  of 
Defense  Handbook  H109,  6  May  I960,  "Statistical  Procedures  for  Deter* 
mining  Validity  of  Suppliers'  Attributes  Inspection,  "  The  purpose  of  the 
paper  is  to  reveal  the  thinking  which  guided  the  statistical  research  per¬ 
formed  and  which  must  form  the  groundwork  for  the  administrative  uses 
to  which  the  Handbook  can  be  put.  hi09  may  bs  applied  to  an  increasingly 
popular  contractual  arrangement  between  consumer  and  supplier  in  in¬ 
dustrial  and  governmental  circles.  This  arrangement  now  forme  part  of 
the  procurement  policy  of  the  Department  of  Defense,  namely,  that, the  ; 
supplier,  prior  to  offering  hie  product  for  acceptance,  will  perform  the 
Inspections  and  taata  necessary  to  ascertain  that  the  material  masts  all 
quality  requirements  established  by  the  ooneumer  and  mada  part  of  the 
contract.  To  do  this,  the  supplier  binds  himself  to  establish  a  system  of 
quality  inspection  over  and  above  his  own  ayetem  of  quality  control,  to  oovar 
all  those  quality  characteristica,  inepaction  procedure*  and  tests  required 
by  the  consumer  to  satisfy  him  that  the  product  meets  all  contractual 
requirements  which  define  materiel  acceptable  to  him, 

One  aspect  just  mentioned  may  be  a  bit  confusing.  It  may  not  Lie 
immediately  apparent  why  the  supplier  should  institute  a  quality  inspection 
eystem  over  and  in  addition  to  his  quality  control  system.  Dias  major 
cojective  in  quality  control  to  watch  tha  effect  of  the  manufacturing  pi >>• 
ccia  on  rnmjyinn"*-  quality  characteristics  of  interest  to  minimise  pro 
auction  of  no. .-conforming  Items.  When  each  material  seems  about  to  be 
produced,  suitable  change*,  are  instituted  in  the  proceee  to  bri"|i  it  back 
into  statistical  control  and  to  continue  production  of  conforming  mate* lb1 
On  the  other  hend,  the  major  objective  of  the  quality  inspection  system 
i  r  to  determine  that,  the  material  manufactured  meet*  all  quality  require¬ 
ments  prescribed  In  the  contract.  In  a  sense,  it  ic  a  measure  of  tnc 
success  of  ths  quality  control  system. 


"‘This  paper  was  presented  at  the  Conference  on  Reliability  Assurance 
Techniques  for  Semi-Conductor  Specification,  and  appears  in  the  Pro¬ 
ceedings  of  this  conference.  We  wish  to  thank  the  editors  of  said  Pro¬ 
ceeding*  for  permission  to  reprint  this  paper. 
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To  return  to  the  policy  of  the  Department  ol  Defatto  end  of  many  in- 
duetrlal  eoneumere,  (or  many  practical  re* eon e,  they  lnqtgt„that  the 
supplier  offer  (or  acceptance  only  material  wuich  he  haa  codn  :&rid  i\e((U 
clent  reason  to  believe  con(orme  with  all  contract  requirement*.  The  only 
way  to  gain  euoh  knowledge  objectively  ie  to  perform  consctentiou  sly  and 
competently  the  various  inspections  end  tests  by  which  the  contract  de> 
fines  product  acceptability.  This,  of  courser  clearly  requires  that  the 
supplier  establish  a  system  of  quality  Inspection  for  his  finished  product 
which  parallels  the  system  which  the  consumer  must  plan  to  assure  that 
the  quality  provisions  of  ths  contract  are  being  honored.  The  supplier 
will  certainly  include  the  cost  of  hie  quality  itiepectlon  system  in  his 
over-all  item  coetj  the  consumer  must  expect  this,  At  the  ■amatltn*,. 
sines  ths  consumer  will  eventually  pay  Its  cost,  h*  has  a  special  right 
and  interest  In  the  supplier's  quality  inspection  system,  oil*  might  say1" 
a  proprietary  interest.  In  e  sense,  the  don  turner  is  buying  not  only  a*:: 
product  but  a  eervlct  ••  the  quality  inspection  system.  The  con  turner 
can  ute  the  supplier's  system  to  reduce  his  own  acceptance  inspection 
cost  and  yet  lose  little,  if  anything,  in  aesurence  of  product  quality, 

This  can  le  done  through  procedures  collectively  called  "product 
verification"  based  on  the  following  philosophy)  If  the  supplier's  quality 
inspection  system  is  competently  established  and  rim  so  that  it  yi  elds 
valid,  objective,  data,  equivalent  in  all  reepeeti to  the  data  thb  q'qnipgiidr 
himself  would  get  by  inspecting  the  seme  lot,  then  the  supplier's  data  ar* 
just  as  trustworthy  a*  a  basis  for  acceptance  as  the  consumer's,  Went1*, 
if  ths  latter  is  assured  that  the  supplier's  quality  inepaction  system  yields 
vaud  r*ita  ,  h*  may  lorthwlth  accept  any  material  the  supplier  offers  for 
he  may  confidently  expect  it  to  Meet  contractual  requirements  were  he 
himsulf  to  inspect  it  fully.  Thi.  assurance  in  obtained  by  the  procedures 
collectively  described  as  produce  verification. 

The  first  step  in  product  verification  is  to  determine  that  the  euppUer 
has  the  organisation  and  physical  means  in  being  or  available  to  him  l 
perform  all  the  Inspections  and  tests  required  by  the  contract  to  determine 
the  acceptability  of  the  product.  Thi*  means  not  only  the  several  gages 
and  p’flt-e*  of  equipment  involved,  but  the  means  to  calibrate  or  etan'^rdiee 
them  periodically  at  requirad. 

Ths  next  step  is  to  dstsrmm*  that  the  supplier  has  k  definite  plan  or 
program  for  using  this  equipment  at  specified  and  that  the  personnel 
assigned  to  do  thie  work  are  competent  in  all  technical  and  administrative 
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phases.  The  coniumer  mult  particularly  be  satisfied  that  the  supplier  ha* 
established  a  reporting  system  that  will  make  available  to  both  parti** 
complete  and  timely  information  a*  to  th*  technical  finding*  of  the  quality 
inspection*. 

Third,  the  coniumer  muit  be  »ure  that  the  supplier'*  quality  inspection 
personnel  and  hi*  own  acceptance  personnel  u*«  the  same  inspection  and 
test  procedures  and  standards  so  that  thay  git  thi  Santa  results  on  impacting 
a  given  sample.  To  avoid  differences,  misunderstandings,  end  disputes 
visual  quality  standards  should  bo  established  to  the  extent  required  so 
that,  ideally,  both  partis*  would  always  agree  as  to  whether  a  given  item 
ia  defective. 

Fourth,  we  enter  upon  validation,  the  field  of  coverage  of  H109,  The 
coneumer  must  perform  his  own  Inspections  and  tests  of  the  material 
offsrod  for  accaptanca.  It  is  not  th*  primary  purpose  wf  this  activity  to 
determine  the  acceptability  of  the  material  inapected,  Rather,  the  o^jaotiye 
is  to  determine  whether  the  result*  obtained  by  th*  supplier's  quality 
inspection  system  are  Valid  and  essentially  th*  same  as  thee*  obtained  by  . 
the  coneumer  and  may,  therefore,  be  used  to  justify  acceptance  of  the 
material  offered,  in  other  words,  the  supplier  offere  for  acceptance  . 
lot*  of  material  which  hit  quality  inspection  system  has  found,  through 
objective  evidence,  to  meet  all  contractual  requirements.  If  his  results 
ere  determined  to  be  valid  by  the  consumer,  then  th*  latter  should  ba 
willing  to  accept  ths  supplier's  inspection  data  as  sufficient  evldsnca  of 
Lite  acceptability  of  the  material.  It  ia  assumed  that  the  consumer's 
l"  ".paction  results  are  a  standard.  If  tha  rssults  generated  by  the  supplier's 
quality  inspection  system  sr*  essentially  tha  same  as  the  standard,  then 
acceptance  mav  fairly  end  properly  be  based  upon  them. 

Ths  essence  of  ths  product  verification  which  Department  of  Defense 
policy  requires  of  the  Government  Inspector,  than,  is  to  determine  tna 
a:  i  Uunce  of  ths  physical  means  end  organisation  for  quality  inspection, 
of  the  necessary  training  and  know-how  on  ths  part  of  the  supplier's 
personnel  using  these  moans,  end  continued  surveillance  of  the  apparatus 
and  procedures  employed.  Validation  is  tha  last  step  and  Is  a  tool  for 
cheeking  the  effectiveness  of  the  entire  system  cf  quality  inopeetton  which 
is  constantly  under  surveillance  by  the  consumer's  inspector,  It  is  plain, 
from  the  breadth  of  responsibilities  necessarily  associated  with  this 
policy,  that  the  consumer's  inspector's  span  of  capabilities  and  competance 
must  be  far  beyond  those  of  the  gsgs-pushing  inspector  of  yesteryear  if 
he  ia  to  be  expected  to  perform  competently  the  duties  laid  upon  him. 
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For  on*  thing,  ha  muat  b*  thoroughly  familiar  with  the  concepts  laid  down 
in  thia  papar  and  equally  with  tha  now  raaponalhilitlaa  ha  bear  a.  Ha  muat , 
recognise  that  tha  specific  techniques  neoeiiary  to  oxocuta  thaaa  respon¬ 
sibilities  may  ba  axpectad  to  vary,  and  tharafor*  muat  ba  developed  anaw, 
in  aach  auppllar'a  plant,  Thia  program  preiont*  a  definite  challanga  to 
tha  old- Una  Impaetor  diractad  toward  tha  up«gra4ihg  of  hla  abilities  In 
tha  modern  age, 

Tha  purpoia  of  validation  ia  to  check  tha  auppllar'a  ayatam  of  impaction. 
Henca,  it  i*  daalrad  to  ehack  result's  on  both  acoaptad  and  rejected  Iptp,  for 
thaaa  will  antar  into  computation  of  tha  procaaa  average  which,  in  MIL,- 
STD  105,  controla  reduced,  normal,  and  tightened  inspection,  It  la  the  1 
conaumer'a  purpoia,  than,  to  uia  hil  validation  raaulta  to  justify  tha  data 
generated  by  tha  auppllar'a  quality  inspection  system;  ha  ia  not  trying  to 
find  whether  to  accept  the  material,  'It  ia  thia  point  which  a  a  am  a  moat 
difficult  for  tha  nawcomar  to  vaUdation  to  grasp.  As  long  a  a  the  auppllar'a 
raaulta  ara  found  Valid,  hia  data  may  properly  be  uaad  to  Justify,  acceptance 
of  tha  matarial  ha  offara  aa  masting  all  raquiramantir :  Whan  tha  consumer's 
raaulta  appaa r  to  differ  aignlfleantly  from  those  of  tha  supplier  ,  aa  indicated 
in  application  of  Ta.bla  I  or  Table  HI  of  Ml 09,  than  tha  consumer'*  repre¬ 
sentative  must  raviawtha  lupplia.  'a  quality  lnapaotiun  ayatam  from  stem  ■ 
to  atarn  to  determine  tha  cause  of  the  discrepancies  noted.  Administratively, 
timvti  aaama  to  bo  little 'point  in  tamng  the  supplier  with  the  fact  that  hia 
chu.  do  not  jibe  with  thoaa  of  the  consumer  unless  h«  can  be  told  why.  In 
this  area,  one  must  kaap  hia  criticism  aonatructiva  but  this  doe*  not 
absolve  the  supplier  from  ilia  responsibility  of  reviewing  hla  own  activity 
onca  ''a  la  told  tha  validity  nf  hia  data  ia  in  question.  1 

Department  ot  Defense  Hanobooh  HI  O')  wa*  prepared  to  furnish  atat<  v- 
tical  tools  wharaby  tha  validity  ot  tha  supplier 1 1  data  may  ba  adiudged  In 
comp»  riaon  with  data  obtained  in  validation  inapactlon.  Since  tha  Hand¬ 
book  ta  Intended  for  uaa  by  impactore  in  tha  field,  it  was  designed  to 
require  a  minimum  of  computation,  both  in  quantity  and  apphlatlcatlon, 
and  the  procedure  a  ware  (impUfled  to  the  degree  poaaibla.  Tha  purpoee 
waa  to  publish  a  procedure  which  would  require  no  previoue  atatlatical 
background  on  tha  part  of  tha  usnr. 

It  la  anumed  in  HI 09  that  the  auppliar  ha*  already  inapactad  and 
tested  tha  lot  in  question  for  ali  quality  characteristics  listad  in  con¬ 
tractual  requirement*  and  that  the  data  generSttd  have  boen  made 
available  in  detail  to  tha  consumer,  It  doe*  not  matter  whathar  tha  lot 
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w««  and  acceptable  or  rejectuble,  In  either  case,  the  consumer  may 
aampl*  and  teat  the  lame  : lot  for  the  purpose  of  verifying  the  validity  of 
the  data  furniahed  by  the  lUpplier.  He  take*  a  sample  of  such  else  that  . 
the  ratio  of  the  aupplier1*  lample  to  hit  l*  r,  where  r  ■  1,  2,  3,  ft,  or  ft, 
a*  ahown  in  Table  1,  H109,  The  conaumer  lnapeota  the  aampl*  and  find* 
a  number  of  defect*  or  defective!,  aay  de>  He  compare!  dc  with  da> 

the  number  of  defect*  or  defective*  found  by  the  aupplier  in  hta  aampl*. 

Thia  companion  1*  made  in  Table  I  which  fttmiehe*  the  limit  for  dc  for 
any  dg  within  the  gi  ven,  igimpl*  ratio*,  r,  If  thia  limit  la  equalled  or 
exceed«‘d;7iHe".f:onauiti*r  Itnay  proceed  on  the  theory  that  the  aupplier 'a  |; 
data  are  invalid  and  that,  therefor*,  hie  quality  inepeetion  eyaUm  ahould 
be  reviewed  to  detect  the. Shortcoming,  reeponelbl*.  If  auch  shortcoming 
la  iounu,  th*  consumer  Should  rajadt'  th*  aupplier1*  data  for  the  lot  in 
question  and  substitute  Via  own  a*  a  baaia  for  daclding  whether,  the, lot  ia  , 
acceptable  or  otherwise.  The  "action" "limit  of  Table  I  Wae  computed  on, 
a  S«p#rc*nt  level  of  etatietical  significance  bo  the  chance  le  excellent 
that  an  "action"  indication  will  result  In.jCUdlng  a  true  physical  discrepancy. 
However,  tha  possibility  eti,U  remain*  that  no  shortcoming  really  aiaiBte. 

In  this  case,  there  i*  ae  much  reason  to  truat  one  let  of  data  aa  tha  other 
but  it  la  parhapa  moat  fair  to  .request  th*  aupplier  to  perform  a  reinapeotlon 
of  th*  questionable  lot  under  the  direct  surveillance  of  the  ooneumer1  ■ 
inspector. 

H109  also  contains  procedures  to  detect  amall  biaeee  or  insidious 
diaorapancie*  which  caua*  relatively  amall  bia*  in  ilia  result*  and, 
therefore,  would  ahow  up  only  in  a  aort  of  historical  review  of  tha  evt- 
uence  from  a  number  of  coniacutlv*  validation*,  Thua,  Tabic  1  indicate* 
discrepant  reeui*?  frwui  a  aingl*  validation,  but  the  tendency  for  diacrepant 
validation  t'%,  suite  to  accunulate  in  a  aaquanc*  of  validation*  la  picked  up 
by  Table*  11  and  111,  H109,  Thua,  from  Table  II  (which  ha*  different 
section*,  one  for  each  value  of  r)  wa  can  obtain  a  so-called  "check  rati..g" 
lor  «*ch  oair  of  dc  and  d(  obaarvad  from  each  validation  pe *'f«rmetl, 

The  check  rating*  obtained  from  Table  II  ar*  equally  valid  when  different 
r'a  ere  ueed  or  for  different  defect*  or  defect  claeaea.  It*  omnibus  nature 
make*  it  quite  flexible  and  valuable,  Th*  check  rating*  ar*  added  together 
and  their  turn  compared  with  the  "median"  value*  and  th*  "warning"  and 
"action"  limit*  given  in  Table  III  for  the  number  of  lota  whoee  check  rating, 
are  cammed.  Th*  median  value  la  what  ie  expected  if  tha  supplier' a  result* 
are  commensurate  with  thoa*  of  tha  eoneumer.  If  the  summed  check  retlnge 
reach  the  "Warning  Critical  Value"  there  la  reason  to  seek  a  discrepancy 
in  the  auppiler'e  quality  inspectim  system.  A*  before,  thia  limit  wa* 
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calculated  at  a  5-percont  level  of  etatiaticxl  eignificance  and  the  action 
to  be  taken  1a  that  already  daaerlbed,  However,  if  the  "Action 
Critical  Value"  li  reached  (at  the  1  "percent  level  of  eignificance)  the 
coneumer  la  juatifled  in  rejecting  the  aupplier'a  data,  whether  or  not 
he  hae  been  able  to  locate  the  aource  o'  trouble  in  the  latter* a  ayatem. 

Whenever  obaerved  differencea  hi  va  been  traced  to  aome  ahorteoming 
in  the  aupplier'a  quality  inapaction  ayatem  and  the. ayatem  her  been  corrected 
to  the  cunaumer'a  eatUfaatlon,  he  can  atart  on  a  now  cycle  of  validation. 

Until  aome  ahorteoming  haa  bean  found  and  corrected,  there  la  no  reaeon 
for  etarting  a  new  cycle. 

Cuita  aaida  from  the  contractual  nature  of  the  agreement,  the  auppller 
muat  alwaya  recogniae  that  .hie  quality  Inapaction  ayatem  la  being  cold 
aa  a  service  to  the  coneumer  and,  al  euch,  muat  aatiafy  him  at  to  the 
propriety  of  lta  atructuro,  procedural,  and  reeulta,  the  validity  of  which 
ie  checked  by  hia  product  verification  activltiaa,  the  auppller  muat 
alwaya  be  willing  to  modify  or  Improve  Hia  ayatem  at  re  quailed  by  the 
coneumer  ao  long  aa  the  validity  of  hia  raaulta  it  hot  affected  hdverialy 
thereby,  The  aupplier'a  willlngneaa  to  pleaao  the  coneumer  atema  from 
recognition  that  the  latter  ia  paying  for  the  ayatem  and  uaaa  the  data  it 
generatca  aa  a  baale  for  acceptance  of  the  product. 


ADDITIONAL  ANALYSIS  OF  MISSILE  TRAJECTORY 
MEASURING  SYSTEMS 


Oliver  L.  Kingsley  «nd  Bernle  R.  Free 
Rang*  Instrumentation  Systems  Office 
Whit*  Sand*  Mlaall*  Rang* 

N*w  Mexico 


I.  INTRODUCTION.  Four  flight  teats  h*va  bean  conducted  for  th# 
study  of  eoouraoy  and  preotalon  of  missile  trejeotory  measuring  systems  at 
White  Sands  Missile  Range.  Date  oolleoted  from  tha  first  two  tests  have 
been  analysed)  Interim  reports  have  bean  published)  and  high  lights  of  the 
results  ware  presented  by  Mr.  Kingsley  at  previous  nnnfarenoes  on  tha  design 
of  experiments.  Date  from  the  third  flight  test  la  In  the  process  of  being  ansi* 
lysed. 


The  purpose  of  this  paper  is  to  present  a  summary  of  the  results  from 
the  third  flight  test  and  to  oompare  them  with  results  from  the  first  two  tests. 
The  four  tracking  systems  to  be  dlsoussod  are  the  ballistic  camera Askanla 
olnatheodollte.  DOVAP.  and  FP8-*  1 6  radar . 

The  ballistic  camera ,  which  Is  a  fixed  oamure,  photographs  a  flashing 
light  on  the  missile  against  a  star-trail  background  The  plat*  from  aeoh  cam¬ 
era  yields  angular  date  and  missile  position  Is  computed  using  the  method  of 
over- determination. 

The  AskanU  oinetherwJoltt*  Is  *  tracking  camera.  It  photographs  tha 
mlsitla  along  with  internal  u'sle  which  show  tha  aelmuth  and  alevatlon  angles 
of  t,h*  optical  axis  for  each  frama  of  the  film.  Poaltlon  data  le  wamnuted  using 
the  over- determination  method. 

DOVAP  le  *  continuous  wav*  electronlo  system  which  utilises  the 
dopplar  principle  to  determine  position  dste. 

Each  rPS-16  radar  mueiures  range  and  aslmuth  end  alevatlon  angles. 
Dele  from  aeoh  radar  aen  be  used  to  estimate  tha  missile  trajectory,  or  data 
from  eevaral  radar*  can  b*  combined  to  give  a  composite  estimate.  The  radars 
from  which  data  is  aombinsd  form  e  composite  radar  system. 

Although  the  term  “eyetem"  Is  sometimes  uaed  in  this  analysts  to  refer 
to  on*  of  the  four  trajectory  measuring  systems,  in  a  broader  sense  It  mean* 
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the  end-to-end  proceai  of  collecting,  converting,  reducing  and  reporting 
position  data.  It  la  tnlhla  broader  aenae  that  the  a  animate  a  ofadcuraoy  and 
preoialon  have  meaning,  In  other  worda,  the  date  on  which  analyses  were 
performed  contain  the  offoota  of  final  data  reduction  and  reporting. 

The  analyala  for  the  third  teat  covered  three  sections  of  the  trajectory 
as  shown  in  Tabla  I.  Data  were  available  from  all  systems,  except  DOVAP, 
for  ail  three  trajectory  saetlona.  Date  ware  available  from  the  DOVAP  system 
for  Section  1  only.  The  bias  error  analyala  for  each  system  will  be  presented 
first,  Then  the  precision  analyala  will  be  discussed.  Significant  improve¬ 
ments  In  system  performenoe  wtll  be  pointed  out. 


TABLE  I 


— 

Saotlon 

Date  Points 
Sampled 

Nominal  Tima  Along 
Trajectory  UaoohtU). 

I 

48 

34  to  60 

II 

44 

101  to  125 

III 

_ j 

16?  to  192 

it.  BIAB  ERROR  ANALYSIS. 


A,  BalllatU  Camera.  When  properly  located  the  balllstlo  otmere 
system  can  yield  vary  good  unbiased  trajectory  data.  For  thla  iwaauu  the 
bslllaUo  camera  system  was  ohoaan  es  the  bias  standard  for  WSMR  and  bias 
errors  in  the  evetam  era  assumed  to  be  sero.  Using  data  from  the  belUftfo 
camera  system,  bias  estimates  forth#  other  systems  weie  obtained  eu  follows: 

A  parameter  measured  by  the  jth  instrumentation  system  at  ths  ith  time 

may  be  expreaied  as; 

xi)"xtl  +  bU  *•» 

Where  x^  *  the  true  value  of  tha  paramater  at  the  ith  time, 
b|j  ■  the  bias  error  of  the  jth  system  at  the  1th  time, 
and  e^j  ■  the,  random  error  of  the  Jth  system  at  the  ith  time. 


* 


Design  of  Experiments 

An  error  for  tha  Jth  system  at  tha  1th  time  la  written  aa 
Xlj  "  XU  “  XIBC 

whara  BC  repraaenta  tha  balllatla  camera  ayatam,  Thta  oan  be 
rewritten  ua; 
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A  xu  -  xu  ♦  by  ♦  atJ  -  (xu  ♦  blBQ  >  e1BC)  «  by  4  etJ.  -  elB0 

ainoa  it  la  aaaumad  that  b,nC  *  0.  Next,  It  la  aaaumed  that  the  jura 
of  the  n  random  error*  aatociateirwlth  tha  n  meaaurementa  made  by  the  Jth 
ayatam  goaa  to  aaro.  Thla  aaaumptlon  la  written'  aa: 

n 

T  e.  »  0.  (J  ■  any  inatrumentatton  ayatam) 

1-1  *3 

Now,  if  tha  Axtj  are  aummed  for  the  n  maaauramonta,  wc  have 
1  b“  *  '  *.8C 


1  > 


}•  «1j 

-SI 

! 

fl 

a 


The  laat  two  tanvi*  on  the  right  go  to  aero  under  the  above  assumption, 
and  tha  equation  becomes  " 


I,a,u  ■  i  b-> 

The  aum  of  the  left  oan  be  computed  directly  from  data  from  "he  tth 
ayatam  end  correaponding  data  from  tha  balllatlc  camera  ayatam,  ft  la  an 
ultimate  of  the  aum  of  tha  n  biaa  errors,  by,  associated  with  the  n  measure¬ 
ments  made  by  the  jth  ayatam.  On  taking  tha  mean  aa  follows 

n  £AxU  ■£  Xl  "  "b> 
l«l  l“l 

the  expression  on  tha  left  yields  «  bias  estimate  for  the  Jth  ayatam. 


« 


B.  Aakania  Cinetneodollte.  Table  II  presents  bias  estimates,  In  terms 
of  cartaaian  components .  obtained  from  the  flrai  three  testa.  Theca  estimates 
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were  computed  using  trajectory  data  from  all  section*.  For  each  component 
the  bias  oslUuates  obtained  from  the  throe  testa  are  alike  In  algn  and  similar 
In  magnitude.  Estimates  for  the  North  end  East  components  are  reasonably 
small,  whereas  they  are  consistently  larger  in  the  Up  component.  Apparently, 
the  corrected  elevation  angles  arh  smaller  than  the  true  elevation  anglea. 

Perhaps  a  better  approximation  of  the  refraction  correction  would  yield  a  desired 
Improvement  of  the  negative  bias  in  the  Up  component. 

For  Test  No.  3  all  three  component  bias  estimates  were  significant  at 
the  5%  level  compared  to  an  expected  value  of  earo.  Also,  the  analysis  of 
varisnco  of  error  data  revealed  a  significant  shift  In  the  magnitude  of  uyatam 
bias  over  Section  I  In  the  North  component,  Section  II  In  all  three  components, 
and  Seotlon  III  in  the  North  and  Up  components. 

C.  DOVAP.  Teat  No.  3  was  instrumented  with  two  DOVAP  systems. 

The  standard  system  operated  at  36.2  mo.  The  second  system,  whloh  operated 
at  36.9  mo,  was  used  to  test  the  Interstate  Transponder.  DOVAP  data  were 
available  In  Seotlon  I  only.  Table  III  prasants  the  component  bias  estimates 
for  the  eystem  for  the  first  three  tests, 

l  '  '  ' 

Each  test  revealed  Improved  system  bias .  Electronic  reeding  and 
digitising  equipment  was  Introduced  Into  the  reduction  process  on  Test  No,  2. 

For  Test  No.  3  signal  propagation  velocities  Wort  estimated  (uslng.  a  table  look* 
■ip  technique  which  took  into  account  the  missile  height  above  the  transmitter, 
These  changes  In  data  reduction  greatly  roduoad  the  system  bias.  'Further  "re- 
Auction  of  system  bias  was  investigated  through  Improvements  in  start  point 
dutermlnttlon  and  by  making  adjustments  for  Him  tulaUv*  locations  on  the 
missile  of  the  flashing  light  (ballistic  camera  system  reference  point)  and  thr. 
DOVAP  ontanna . 

The  bias  estimates  for  1>st  No.  J  were  ail  significant  at  the  5%  level 
mmared  to  an  expected  value  of  zero.  Also,  there  was  o  slgnltuunt  Nhtft  In 
th*  magnitude  of  system  bles  over  the  section  In  the  North  and  East  component 

D.  FPS-16  Radar.  Bias  estimates  for  three  of  the  FPS-  1ft  radars  are 
shown  In  Table  IV,  All  of  the  redars  operated  In  >he  sxln  track  mode.  The 
estimates  are,  in  general,  larger  than  those  for  the  Askanla  and  DOVM  systema. 

The  bias  estimates  for  R-112  ere  smeller  for  Tests  No.  1  and  3  then  for 
Test  No.  2.  They  range  In  magnitude  form  2  to  S9  feet.  For  R-114  the  estimates 
are  very  good,  exoept  for  the  66  feet  in  the  East  component  of  the  Test  No.  3, 
They  range  in  magnitude  from  0  to  12  feat,  Radar*  R-112  and  R- 114  era  both 
located  at  the  southern  end  of  the  range  end  tracked  the  missile  as  It  moved 
north . 
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the  bias  estimates  for  R— 122  are  more  consistent  in  size  lor  the  three 
tests,  partly  beaause  of  Its  loaatlon  near  mid-range.  On  the  average  It  was 
closer  to  the  trajectory  sections  analyaad  than  either  R-112  or  R-114.  The 
estimate*  isnge  in  magnitude  from  2  to  34  feet. 

Table  V  presents  the  bias  estimates  for  a  composite  trajectory  obtained 
by  averaging  the  respective  components  of  all  three  radars.  These  estimates 
are  very  similar  for  those  of  R-122. 

With  allowances  made  for  the  skin  track  mode  most  of  the  radar  bias 
estimates  for  the  North  and  Up  components  were  significant  at  the  5%  level 
compared  to  an  expected  value  of  zero.  Also,  there  were  significant  shifts  In 
the  magnitude  cf  system  bias  in  all  three  components  over  Seotion  I  for  R-112 
and  R-114,  and  In  the  Up  component  over  Seotion  III  for  R-122. 

Slnos  their  Installation  at  W8MR  tha  radar  systems  have  been  used 
extensively.  A  series  of  calibration  and  evaluation  tests  have  been  proposed! 
these  tests  have  bean  planned  for  tha  near  future. 

III.  PRECISION  ANALYSIS.  Precision  la  defined  as  a  measure  of 
variability  of  a  random  variable  about  Its  mean  value.  For  this  analysis  it 
is  synonymous  with  tha  statistical  term,  standard  deviation.  Three  msthcda 
ware  used  to  obtain  precision  estimates  for  the  four  trajectory  measuring 
systems.  A  brief  description  of  aaoh  method  follows.: 

1.  Over-determination  Point  Estimate  of  Praolrlon  •  Ovei-deteimlnatlon, 
*y  the  method  of  least  squares,  of  space  points  on  *  uejeotory  yields,  as  a 
0.1.1b  product,  variance  estimates  for  teoh  space  point,  When  the  variance 
estimates  for  a  sample  r'i  space  points  ere  pooled  the  result  is  a  point  aatlmate 
cf  preuiulon.  A  major  disadvantage  of  thle  method  is  that  it  is  sensitive  to 
system  bias  errors  In  the  Input  data.  However,  for  thi*  analysis  there  ere 
ways  to  Isolate  a  biased  system  for  inveitigetion. 

2.  Multi-Instrum«.nt  Estimate  of  Precision  -  This  method,  sometimes 
referred  to  as  the  Simon-Grubbs  method,  requires  a  simultaneous  sample  of  ' 
trajectory  space  points  from  eeoh  of  three  or  more  Instrumentation  sy^tuM.. , 

Tha  method  yields  a  precision  estimate  for  each  system  Involved.  It  is  insen¬ 
sitive  to  constant  system  bias  errors  in  the  input  data,  but.  shifting  system  bias 
errors  will  enlarge  the  estimates  produced  by  this  method. 

3.  Variate  Difference  Estimate  of  Precision  -  Suocesslvs  differencing 
of  a  sample  of  trajectory  space  points  determined  by  a  single  system  proceed* 
until  the  systematic  elements  of  the  data  become  nenltqtble  end  the  randcin 
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element  oeoomes  dominant.  Thu  precision  estimate  for  the  system  is  based  on 
the  residual  random  element.  This  method  requires  a  data  sample  with  points 
equally  spaoed  in  time  and  ascumes  the  data  can  be  approximated  by  a  polynomial 
All  linear  trends,  which  are  first  degree  elements,  are  filtered  out  and  thus  the 
precision  estimates  by  this  method  tend  to  be  smaller  than  those  obtained  by 
the  two  methods  mentioned  above. 

A.  Dalllatia  Camera.  Precision  estimates  for  the  ballistic  camera 
system  are  shown  In  Table  VI.  The  estimates  ware  obtained  by  a  multi-instrument 
method  whore  variances  ware  pooled  over  the  three  trajectory  seotlons.  The 
precision  estimates  are  tan  fsat  or  loss  in  magnitude  and  similar,  for  the  three 
teste.  These  estimates  correspond  to  an  average  ballistic  camera  angular 
precision  of  10"  of  arc.  The  new  BO-4  ballistic  cameras,  of  which  four  have 
boon  installed  at  W8MR,  are  designed  for  an  angular  precision  of  1"  of  aro. 

This  will  lower  the  precision  estimates  by  o  factor  of  ten.  Hopefully,  the 
BC-4  system  will  beoome  the  standerd  for  the  range  In  the  near  future. 

B.  Askenla  OUatheodollto .  Table  VII  presents  precision  eetlmetea  for 
the  Aakenle  olnetheodollte  system  for  the  first  three  tests.  These  ere  also 
multi- instrument  estimates  with  the  variances  pooled  ever  the  seotlons,  In 
general,  thser.  eetlmetea  ere  about  twice  as  large  at  those  obtained  for  the 
balllstto  camera  system.  They  are  similar  for  the  throe  taste  end  oorreapond 
to  a  system  angular  precision  of  approximately  3ti"  of  aro.  Thesa  praoislon 
estimates  agree  very  well  with  those  computed  for  the  oystem  over  a  number 

of  years  of  operation. 

C.  DOVAP.  Preutp'uri  estimates  for  the  DOVA*  system  are  shown  iri 
Table  VUl.  Thane  w<*n«  obtained  by  the  variate  difference  technique.  Except 
for  Test  No.  2  the  asttmatdsare  lu-o  than  0.6  feet.  They  are  nomparable  to 
precision  estimates  expected  from  the  BC-4  ballistic  camera  eyatam.  Tor  nil 
three  tests  the  DOVAP  has  been  the  most  precise  system  at  W8MR.  The  multi- 
instrument  method  was  also  used  to  obtain  estimates  of  precision  for  this 
system.  However,  some  of  the  variance*  were  negative  in  sign.  Since  these 
aro  less  meaningful  for  the  purpose  Intended  ana  more  difficult  to  lntnrpruL,  they 
are  not  Included  in  this  paper. 

D.  F PS- 1 6  Radars.  Precision  estimates  for  three  of  the  FP8- 16  radars 
are  shown  in  Table  IX.  These  are  multi- Instrument  eettmata*.  Variability  in 
the  udars  is  considerably  larger  than  in  any  of  the  other  eystems.  Generally, 
it  increased  over  the  three  tests,  especially  for  R- 112  in  the  East  and  Up  com¬ 
ponents.  The  estimates  for  R- 114  are  smaller  than  those  for  R- 112 .  The 
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largest  change  is  in  the  Up  component  for  Test  No.  3.  For  R-122  the  precision 
estimates  are  more  consistant  than  for  the  other  two  radars.  Bias  estimates 
for  R-122  also  had  this  characteristic. 

Radar  system  cartesian  component  variability,  shown  in  Table  IX,  must 
be  viewed  in  light  of  the  variability  of  the  measured  parameters,  namely  range 
and  azimuth  and  elevation  angles.  The  computed  .vrtesian  components  depend, 
in  various  degrees,  on  these  measured  parameters.  For  instance,  for  both  R— 112 
and  R— 114  the  range  measurement  is  essentially  a  measurement  of  the  North 
component,  whereas  the  azimuth  a.nd  elevation  angle  measurements  account  mostly 
for  the  computed  East  and  Up  components  respectively.  This  relationship  arises 
from  the  fact  that  both  R- 112  and  R— 114  are  located  at  the  southern  end  of  the 
range  and  the  missile  moved,  in  general,  North.  For  R-122  the  relationship 
is  not  so  evident  since  it  is  located  near  mid-range.  Table  X  presents  preci¬ 
sion  estimates  in  terms  of  measured  parameters  for  the  three  radar  systems. 

A  comparison  cf  Tables  IX  and  X  shows  that  the  precision  estimate  of 
154  feet  in  the  East  component  for  R— 112  on  Test  No.  3  corresponds  to  the 
large  precision  estimate  (0.55  mils)  obtained  for  azimuth  measurements  made 
by  this  radar.  Likewise,  the  estimate  of  73  feet  in  the  Up  component  for  R-114 
on  Test  No.  3  corresponds  to  the  estimate  obtained  for  elevation  measurements 
for  this  radar.  The  precision  estimates  of  Table  X  reveal  more  directly  the  per¬ 
formance  of  the  radars. 

Following  the  installation  of  the  first  FPS-16  radar  in  1958  a  series  of 
evaluation  tests  were  conducted.  In  a  report*  covering  these  tests  the  general 
conclusions  were  that  the  exDected  variability  cf  the  radars  would  be  on  the 
order  of  in  y«rHc  jr  range  and  0;2  mils  in  azimuth  and  elevation.  The  range 
precision  estimates  of  Tabic  X  meet  this  expectation.  One  might  suspect  this 
large  angular  variability  to  be  caused  by  glinting  since  the  radarc  ooera'ced  in 
thv?  skin  track  mode.  However,  for  the  three  tests  being  considered  the  tno.d- 
mum  comoonent  in  the  precision  estimates  attributable  to  glinting  ;■=.  only  about 
0,09  mils  and  is  therefore  considered  negligible.  As  stated  at  the  section  on 
bias  errors  a  series  of  calibration  and  evaluation  tests  h3ve  been  planned  for 
the  radars.  Hopefully,  the  angular  variability  will  be  reduced. 


1  See  Reference  6 
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PRECISION  ESTIMATES  FOR  FFB-16  RADARS 

Radar 

Flight 

Standard  Deviation  for  Measured  1 

Tost 

. . . ...Pa 

re  meters  . _ 

Range 

Aelmuth 

Elovetlon 

(vds) 

(hills) 

. ....  (mils) 

R-112 

1 

2.7 

0*42 

0.26 

2 

4.2 

0.37, 

0.52 

3 

7.3 

O.SS 

■  0.31 

R-114 

1 

3.2 

0.17 

0.  IS 

2 

4.8 

0.26 

,.  0.21 

3 

4.2 

0.21' 

0,33 

R-122 

t 

1.4 

0.20- ' 

0.26 

2 

2.6 

0.23 

0,13 

3 

7.8 

0.60 

0.26 

JV.  tttMMMK .AMP.. gattamiflltf. 

A.  The  DCVAF  continues  u.  be  the  most  preolne  trajectn.y  meeeuring 
system  et  WSMH.  With  improvements  In  techniques  of  itert  point  determina¬ 
tion,  end  with  continued  refinement  of  propagation  velocity  eettmatea,  the 
DOVAP  can  also  become  one  of  the  leaet  biased  syetema, 

i,  The  present  ballistic)  oamsra  system  Is  the  second  most  precis* 
system  at  WSMR.  The  new  BC-4  system  Is  expected  to  be  ten  times  more 
precise  than  tho  present  system. 

C.  The  angular  precision  of  the  Askanla  olnetheodolltce  Is  estimated 
et  36"  of  ere,  A  large  bias  still  exists  In  the  Up  component,  flatter  approxi¬ 
mation  of  the  refraction  correction  will  probably  Improve  elevation  determination . 

D.  The  radars  did  not  perform  as  wall  aa  was  expected.  System  blaa 
needs  to  be  reduced.  Also,  improvements  are  needed  In  system  precision, 
especially  In  angular  data. 
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Aspects  to  control  liquid  propellant 

SLOSHING  BASED  UPON  EXISTING  THEORY  . 

W,  R,  Eullta 

Marshall  Space  Flight  Center 
Hunts  villa  >  Alabama 

SUMMARY,  Liquid  propsllants  uiually  carried  by  a  launch  vehicle 
In  cylindrical  containers,  represent  one  part  in  a  complex  of  several 
coupled  epring  syitems  (missile  bending,  bulking,  etc. ),  By  external 
forces,  the  liquid  epring  system  can  be  excited  to  heavy  liquid  mass  motions 
which  can  be  detrimental  to  the  performance  of  the  launch  vehicle,  The 
vehiole  control  motion  may  be  oonsldSred  the  most  influential  factor  for 
exciting  liquid  oscillations, 

During  the  last  decade,  many  attempts  have  been  made  to  ascribe 
the  response  of  a  liquid  to  exciting  oscillations,  Although  principally  a 
nonlinear  problem,  the  theory  hat  been  confined  to  the  first  order  terms 
only,  due  to  mathematical  difficulties,  This  linear! aed  theory  has  bean 
discerned  in  satisfactory  agreement  with  many  experimental  results  at 
least  as  far  as  the  first  liquid  resonance  is  concerned  which  likewise  is 
the  most  adverse  condition  for  the  vehicle  control  system, 

The  interpretation  of  the  existing  theory  leads  to  dimensionless  para¬ 
meters  which,  Incorporated  in  a  nomograph,  provide  quick  orientation  on 
liquid  behavior  under  varying  oscillatory  conditions.  Such  data  hel'i  to 
define  critical  vehicle  flight  periods  and  to  predetermine  proper  design 
narametsrs. 

The  survey  of  the  parrmiatere  of  oscillatory  liquid  motion  suggests 
possible  means  of  suppressing  liquid  sloshing,  The  pros  and  owns  of 
•  sveral  methods  are  dieouesed  and  furthermore,  tho  prospect, s  oioper 

measurements  of  liquid  surface  motion  aru  indicated  in  the  paper, 


ASPECTS  TO  CONTROL  LIQUID  PROPELLANT 
SLOSHING  RASED  UPON  EXISTING  THEORY  . 

W,  R.  Eulita 

Marshall  Spsiej  flight  Center 
Ilunlsvllle,  Alabama 

SUMMARY,  Liquid  propellants  usually  carried  by  a  launch  vehicle 
in  cylindrical  containers,  represent  on*  part  In  a  complex  of  aavaral 
coupled  aprlng  ayatama  (mlaall*  banding,  bulking,  etc,),  #y  external, 
forcai,  the  liquid  aprlng  ayatnm  can  be  excited  to  heavy  liquid  mail  motion* 
‘vhlch  can  ba  detrimental  to  the  performance  of  the  launch  vehicle,  The 
vehicle  control  motion  may  be  considered  the  moet  Influential  factor  for 
exciting  liquid  oecillationi, 

During  the  laat  decade,  many  attempts  have  been  mede  to  ascribe 
the  response  of  a  liquid  to  exciting  oscillations,  Although  principally: a  " 
nonlinear  problem,  the  theory  has  been  confined  to  the  first  order  terms 
only,  due  to  mathametlual  difficulties,  This  linearised  theory  has  been 
discerned  in  eattafactory  agreement  with  many  experimental  result*  St 
lsast  as  far  as  ths  first  liquid  resonance  le  concerned  which  llkewiee  ie 
the  moet  advene  condition  for  the  vehicle  control  syetern.  '  11 

■ ; 

The  Interpretation  of  the  exleting  theory  leads  to  dimenelonlese  para* 
maters  which,  incorporated  In  a  nomograph,  provide  quick  orientation  on 
liqui  d  behavior  under  varying  oiclllatory  conditions.  Such  data  he!  o  to 
define  critical  vehicle  flight  period*  and  to  predetermine  proper  design 
narametsrs, 

The  survey  ot  the  parameters  of  oscillatory  liquid  motion  suggests 
posaible  means  of  suppressing  liquid  sloshing.  The  pros  and  cone  of 
•everal  method*  are  diecueeed  and  furthermore,  the  proepeuis  ,'*r  Dipper 
measurements  of  liquid  surface  motion  are  indicated  in  the  paper, 
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Symbol 

DEFINITION  OF  SYMBOLS 

Definition 

> 

s 

Liquid  amplitude 

■t 

Liquid  amplitude  under  aplaah  condition 

X. 

Forced  amplitude  of  tranavaree  container  motion 

W 

Forced  frequency  In  rad/ tec 

“>t 

Flrat  re aonant  frequency  in  rad/ aec 

i 

Forced  frequency  in  ope 

<1 

Flrat  ?e aonant  frequency  in  opa 

■  ? 


.  M.  ,  i 

*<  W,  1, 

a 

-vx 

Radlua  of  container 

dw'r.e 

Diameter  of  container 

r,  b 

lroiar  coordinate  a  of  polnta  of  the  liquid  aurfaoe 

H 

Longitudinal  acceleration  of  container 

'!<  £  > 

Beaaal  functlona  (B.  F,  ) 

Zero'a  of  flrat  derivative  of  A,  F  of  fir et  order 
and  flrat  kind 

n  »  ilL- 

Acceleration  ratio  (ge  ■  ace.  due  to  gravity) 

Bo 

<  ■  tanh  2  cc  h 
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h  Filling  height  of  liquid  tri  tho  container 

F  Force  exerted  by  the  liquid  toward  the  tank  wall 

W  Weight  of  liquid  in  the  container 


INTRODUCTION .  In  recent  yeare,  eomprehenitv*  etudlea,  experi¬ 
mentally  and  theoretically,  clarified  the  reeponee  of  a  liquid  under  foroed 
vibration!  to  iuoh  an  extent  that  the  effect  of  eloeh  motion  le  predictable 
.  y  theory  under  ilmplifled  condition!,  Never  thole  11,  the  application  of 
the  theory  for  practical  purpoiei Imi  aiwaye  been 'a-  difficult  undertaking, 

Ai  the  ooneequenae  of  theie  difficult!#!,  connected  with  tome  ikeptioiim 
in  the  dependence  on  theoretical  prediction!  in  thli  particular  field,  extreme 
aonietvatlim  in  vehiole  decign  ha*  lometimee  bean  praotieed  ai  the  alter • 
native, 


Thi.i  uncertainty  itlmulated  a  etudy  on  the  limitation!  of  the  exlitlhg 
theory  and  on  the  effect;! veneea  of  the  perimeter!  ihvoWed’in. ptdwr  ,'to  dig* 
earn  the  praetloai  poiilibHitiei  of  iuppr!!*ing  liquid  itotfh 'motion  and  their 
iippllcablUtlae  with  reepeet  to  eontalnor  configuration  and  flight  condition, 
The  remit*  of  thie  etudy  are  preientad  in  thli  paper, 


'■•brn: 


THU  B 

m 


TOiel»HffB»SKHM 


Hi 


_ _ j^ySe^rbwem  ofliqtmf  oeclllatUm* 

i ■  principally  o  noulhudr  .vrublernlike  all  problem*  of  hydrodynamic e. 


Due  to  mathematloel  difficultly*,  tlx  theory  haa  been  llnearlaed  in  order 
to  eolve  the  problem.  The  unceiUntion  of  the  theory  1»  pei-mlelible  if  the 
exciting  acceleration  of  the  eyitem  (x0  CO*>  l*  very  email  ccmparwd  ‘ r 


•■he  longitudinal  acceleration  (gn)  of  the  liquid  container,  ties  R*f,  1  throuth 
41.  Thie  le  one  limitation  r>i  the  exlutlng  theory, 


With  theie  aiiumptioni  (email  acceleration  of  excitation  and  Hr..,* 
nrleation  of  the  theory),  It  ha«  been  poeeibln  tn  deduce  the  equation  ol 
the  reeonant  frequenelei  of  the  Liquid  eyetern  in  a  flat  bottomed  clroular 
cylindrical  container  and  the  equation  of  the  liquid  eurface  under  foroed 
lateral  ainueeidet  oecitletione, 
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The  general  form  of  the  equation  of  the  liquid  surface  In  motion  le 
(Ref.  1  and  5) 


x„ 


a  0 J 


2 


-  £ 


ft  ■  0 


— T 

/”'  n 

\  ~*r 

(O* 


1) 


coa0  coa  oat 


The  surface  equation  ascribe*  the  elevation  (  ^  )  of  any  point  of  the  liquid 
surface  with  polar  coordinate*  (r,  $  )  at  any  Instant  of  the  cycle  (coa  fait) 
above  the  aero  level. 

I  '  • 

From  this  theory,  a*  well  as  from  the  experiment,  it  follow  rigorous¬ 
ly  thati 

(1)  The  maximum  amplitude*  of  liquid  occur  In  the  plane  of  motion 
at  tha  tank  wall  during  the  first  period  of  resonance  (wave  length  about  8 
tank  diameters).  At  higher  resonances  tha  maximum  liquid  amplitude  a 
de  .'ul.'p  at  the  interior  of  the  liquid  surface  (wave  length  smaller  than  one 
tank  dlamatsr). 

(2)  The  largest  liquid  amplitudes  rver  possible  In  an  oscillating 
Hq-iio  system  also  occur  during  the  first  resonance  period. 

(1)  T)t«  Ufgeei  uquld  amplitude*  ere  reached  when  the  longitudinal 
acceleration  of  lUe  liquid  (  is  eouul  lu  tlni  counteracting  acceleration 

duo  10  gravity  (g0)  or  thrust  of  the  vehicle  (ng0).  Under  this  condulcn 
the  Uquld  atarti  violently  splashing  (slosh  or  eplaeh  condition,  see  Ref.  1 
and  1),  This,  simultaneously,  le  the  upper  limitation  of  the  validity  of  the 
surface  equation  following  from  the  linearised  theory  (Ref,  1). 

(4)  The  lower  limitation  of  the  theory  concerning  maximum  liq  <lO 
amplitudes  is  beyond  any  practical  consideration  as  discussed  In  detail  In 
Ref,  l  (very  large  exciting  amplitudes). 

(5)  The  theory  la  strongly  valid  for  flat  bottomed  cylindrical  con¬ 
tainers  only, 
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Sine*  the  maximum  liquid  amplitude*  (co*«ot  *  1)  at  the  container 
wall  (r  >■  a  *  d/2)  in  the  plan*  of  motion  (co*  0  ■  1)  during  the  firat 
reionant  period  (U>< )  are  of  moat  practical  interest  becaute  of  the  force* 
exerted  toward  th*  tank  wall,  and  if  the  frequency  ratio  aquared  i*  expre**ed 
by  the  Greek  letter  A  ■  ui2/  oJ2  ,  W  designating  the  forced  frequency, 
the  liquid  surface  equation  (implifle*  (Ref,  4) 


•‘here  <  identifies  the  hyparbolic  tangent  term  which  depend*  oti  the  fil¬ 
ling  height  -  diameter  ratio  h/d,  The  term  approach**  unity  if  h£  <1. 

The  llould  start*  splashing,  it  th*  acceleration  of  liquid  along  th- 
tank  wall  (  Co 2 )  1*  equal  to  the  acceleration  due  to  gravity  (go)  or 
thrust  (gn  ■  n  •  g0),  as  pointed  out  earlier,  or  (Ref,  4): 
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C  .  nf°  .  JL*b_ 

•  ttt-  rxr 


co  ^  A.  « cc  A 


a 

K 


and 

(3) 


7*T  ‘ 


*0  < 


r 


The  general  equation  for  the  net  forget  on  the  tank  wall  whloh  la 
the  integration  of  the  pressure  distribution,  (Ref.  5  and  6),  also  simplifies 
if  considered  under  the  most  critical  conditions  (splash  condition)  and 
the  assumptions  above,  Then,  forces  (F)  can  be  expressed  In  terms  of 
the  total  propellant  weight  (W)  according  to  the  following  equation  (sse 
Appendix): 


U>  JL 
w 


1 


2  oi2  A 


&  + 


*o  K 1 


2d, 

*  f 


Equ»-.one  (1)  through  (4)  represent  the  fundamental  formulation  of  liquid 
propellant  response  to  sinutonul  tank  oscillations  under  th*  fallowing 
condition*  of  practical  importance:  Maximum  liquid  amplitudes  (cos  ojv 
«  l)  in  the  plane  of  motion  (cos  u  »  1)  at  the  tank  wall  (r  *  a)  during  the 
first  resonant  period  )  under  forced  vibrations  (x0  u)*)  in  a  cylindrical 
flat  'jcttomed  container  (d)  under  varying  conditions  (n,  K  )  and  Hucaieratlon 
equilibrium  ( (^CO^  %  ng0).  Theeu  equation*  cover  all  container  tlr.ea  and 
all  flight  condition*. 

Equation  (1)  indicate*  that  any  frequency  ratio  X  i*  coupled  with 
a  epeciel  parameter  /x0  K  which  may  be  deeignated  the  "liquid  ampli¬ 
tude  coefficient".  Knowing  thie  coefficient,  the  actual  liquid  amplitude 
f  can  be  concluded  by  multiplying  £/x 0K  with  the  excittng  amplitude 
x’  and  the  term  /(which  ii  a  function  of  the  filling  height. 
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luquation  (3)  ehowe  that  each  parameter  pair,  C,/x0K  end  X  i, a  the 
■plash  parameter  pair  for  a  "design  parameter"  d/x^  which  satieties 

this  equation.  This  consideration  leads  t"  a  nomograph  (Figure  1)  which 
hae  been  explained  in  detail  in  an  earlier  report  (Ref,  4).  The  Correspond* 
ing  parameters,  tlxJC  and  X,  according  to  equation  { 1 ),  are  plotted 
versus  the  corresponding  design  parameter*,  d/xe  X2-  following  from 

equation  (3). 

For  a  container  of  de&10m  (■  400  In)  filled  to  a  height  larger  than 
the  diameter  d(  X  «  1)  and  oscillated  with,  an  exciting  amplltud*  x0-arflC,  ,r 
cm  (a  4  in),  the  design  parameter  would  be  d/x0X2  ■  100.  The  homdgraph 
(Figure  1)  indicates  for  d/x0X*  ■  100  an  amplitude  coefficient  of  Q;*9 
m  25,  At  this  condition  the  liquid  starts  splashing  if  the  frequency  ratio; 

Rf  mj/ti  (square  root  of  A  )  is  about  0.  975,  The  same  situation  would 
exist  tor  a  tank  with  d«*50  cm  (>  20  in)  operated  with  an  exciting  ainjilltuds 
xos^0.  5  cm  («  0.2  in).  In  the  first  case,  the  actual  liquid  amplitude  Would 
he  C  •  29  *  4«**2. 95  m  («  116  In)  inthe  eecond  case  C  ■  29*  0.  2iWl4i,?  cr.t 
(«  5.  8  in).  If  the  frequency  ratio  in  both  cases  i*  smeller  than  0. 975,  then 
the  liquid  surface  awing*  smoothly;  if  the  frequency  ratio  l*  larger  then 
0.  975,  the  liquid  1*  violently  splashing. 

In  Figure  2  the  K  value*  era  tabulated  for  different  filling  height 
ratio*  h/d  and  the  nomograph  give*  the  flret  resonant  frequoncl-e  fj  for 
any  flight  condition  (right  side  double  scale)  in  dependence  of  the  parameter 
d/n  X  according  to  equation  (2),  A  straight  line  from  the  point  indl on 
Ine  exciting  frequency  f  (lvfl  scale)  to  the  first  resunan*  frequency  f^  of 
the  system  (right  scale)  provides  the  frequency  ratio  Rf  ■  f/fj  (intersection 
with  central  scale).  With  this  valuo  the  amplltuds  coefficient  ’/x.K  for 
thie  particular  condition  can  be  found  from  the  nomograph  of  Figure  1, 

In  a  el /t. far  way,  the  largest  force*  possible  on  the  tank  wall  oi  an 
oscillating  liquid  system  (eplaeh  condition)  arc  plotted  in  percentage  of  the 
momentary  propellant  weight  (F/W)  in  Figure  3  versus  the  deeig*  para¬ 
meter  d/xe  X2  for  different  filling  ratio*  h/d  according  to  equation  (4). 

DESIGN  CRITERIA  FOR  LIQUID  SLOSH  CONTROL  FOLLOWING 
FROM  THEORY,  With  the  nomographs  discussed  above,  the  response 
of  liquid  propellant  to  any  vehicle  flight  condition  can  be  estimated  with 
fair  approximation,  From  the  fundamental  equations  (1.  2.  end  3),  is 
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well  as  from  the  nomograhp  Figure  1,  it  follows  immediately  that  the 
liquid  amplitude  coefficient  C/x-0K  increases  with  increasing  design 
parameter  d/xQ  .  The  design  parameter  can  increase  either  with  the 
tank  diameter  d,  or  with  decreasing  exciting  amplitude  xQ  or  decreasing 
filling  ratio  h/d  (  fc).  However,  since  the  amplitude  coefficient  curve  is 
almost  a  straight  line  at  least  for  the  higher  design  parameters  (see  nomo¬ 
graph  Figure  1),  the  effect  ol'xQ  and  on  the  actual  liquid  splash  ampli¬ 
tudes  ^s  (highest  liquid  amplitudes  possible  in  the  system)  i3  small  be¬ 
cause  the  amplitude  coefficient  is  approximately  proportional  to  the  pro¬ 
duct  Xq  K.  .  Thus,  the  most  efficient  factor  on  the  design  parameter  is  the 
tank  diameter  d.  The  h.rger  d  the  larger  are  the  liquid  amplitudes  to  be 
expected  in  the  system,  and  consequently,  the  larger  the  forces  on  the 
tank  wall. 

The  nomograph  Figure  1  also  shows  that  for  design  parameters 
d/xQ  70  the  frequency  ratios  Rf  =  f/f ^  are  very  close  to  unity 

(Rfi  0.965).  The  differences  in  the  critical  frequency  ratios  (splash 
condition)  are  small  among  large  container  design  parameters.  This 
means,  a  slight  increase  of  x0  which  reduces  the  original  d/x0  would 

suddenly  create  the  detrimental  splash  amplitudes  because  the  critical 
f-  eoaency  ratio  under  the  varied  condition  is  smaller  than  the  applied 
frequency  ratio. 

This  suggests  that  frequency  larger  than  0,  9  should  be  avoided  in 
any  case.  A  frequency  ratio  of  Rf  =  f/ff  =  0.  85  may  be  considered  "safe" 

for  ail  practical  cases  in  order  to  provide  a  smooth  oscillation  of  the  liquid 
level.  As  the  nomograph  Figure  1  indicates,  such  a  frequency  ratio  cor¬ 
responds  to  a  design  parameter  of  cl/:-:0  /£.“  =  11.  3.  The  propellants  in  a 
container  of  d^rlO  m  400  in)  would  start  splashing. (highest  liquid  amp’-- 
tude  possible)  at  this  frequency  ratio  if  the  exciting  amplitude  would  be 
Xq-^90  cm  (=  35.  4  in);  (d/xQ  ^  =  400/ 35.  4 10:0.  9  =  U.  3;-  /<  assumed 

unity).  However,  such  an  extreme  exciting  amplitude  is  very  unlikely  in 
practice.  For  a  2.  5  m  (=  100  in)  container,  the  exciting  amplitude  x„  would 

be  about  22.  8  cm  {=  9  in)  to  achieve  the  splash  amplitude  under  the  same 
extreme  conditions. 

Fortunately,  during  vehicle  flight,  the  frequency  ratio  decreases 
automatic  ally  if  the  exciting  frequency  is  maintained  constant.  It  follows 
from  equation  (2),  and  the  nomograph  Figure  Z  shows,  that  the  first  resonant 
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frequency  CO  j  (fj)  increases  with  increasing  vehicle  acceleration  (n). 

Thus,  the  frequency  ratio  A  decreases  and  the  liquid  amplitudes  decrease 
too  (Equation  1  and  nomograph  Fig  1).  The  decreasing  height  of  the  liquid 
(  K.  <C  1)  during  drainage,  however,  increases  X  again  so  that  splashing 
toward  the  end  of  the  powered  flight  is  very  likely. 

■ 

On  the  other  hand,  the  natural  frequencies  of  the  liquid  system  (first 
resonance)  are  small  for  large  diameter  containers;  they  are  inversely 
proportional  to  the  spqare  root  of  the  diameter  d  (Equation  2  and  nomo¬ 
graph  Fig  2),  This  effect  is  aggravated  if  the  vehicle  acceleration  ratio 
n  is  small  (upper  stages).  It  is  easy  to  understand  that  small  resonant 
frequencies  are  easier  to  approach  by  any  vehicle  motion  than  are  larger 
resonant  frequencies.  In  other  words,  liquids  in  large  diameter  containers 
are  more  sensitive  to  any  movement  than  those  in  small  diameter  containers, 
or,  slosh  control  by  vehicle  control  frequencies  is  more  difficult  to  main¬ 
tain  if  the  tank  diameter  is  large  and  if  the  vehicle  acceleration  is  low 
(near  zero  g). 

At  higher  exciting  frequencies  the  situation  changes.  Figure  1  shows 
the  pattern  of  the  curve  of  liquid  amplitude  coefficients  versus  higher 
frequency  ratios  X  ,  the  peaks  indicating  the  resonances.  The  other  curve 
intersecting  the  amplitude  curve  at  the  beginning  represents  the  splash  con¬ 
dition  for  a  particular  parameter  d/xc  .  This  graph  illustrates  that, 

at  higher  resonances,  splashing  of  liquid  starts  before  the  theoretical 
amplitudes  of  liquid  are  obtained.  This  means,  at  higher  resonances, 
there  is  always  splashing  and  the  theory  according  to  Equation  (1)  is  in- 
•  ffective.  No  theory  exists  yet  which  ascribes  the  liquid  surface  undvi- 
these  conditions.  The  only  fact  we  know  is  that  liquid  amplitudes  at  this 
stage  are  limbed  by  the  equilibrium  of  the  accelerations  acting  on  the 
liquid  (splash  condition). 

Another  important  fact  concerning  the  container  design  follows  .from 
figure  3.  It  shows  that  the  force  -  ratio  increases  considerably  with 
decreasing  b/d  -  ratio  for  a  particular  design  parameter  d/x  K2.  This 

e.fect,  of  course,  is  fairly  compensated  by  the  decreasing  propellent 
weight  during  the  drainage  process.  However,  if  an  equal  propellant  volume 
is  considered  in  two  different  tank  configurations,  first,  in  a  long  but 
small  diameter  tank,  and  second,  in  a  short  but  large  diameter  tank,  it 
can  be  concluded  from  Figure  3  that  the  forces  the  tank  wall  in  the  first 
case  are  much  smaller  than  in  the  second  case.  This  again  suggests  the 
design  of  long  but  small  diameter  containers  rather  than  short  but  large 
diameter  containers. 
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PRACTICAL  METHODS  TO  CONTROL  LIQUID  SLOSH  MOTION, 

The  interpretation  of  the  theory  of  liquid  slosh  motion  during  the  first 
resonance  period  in  the  previous  section  already  illustrated. that  the  basic 
container  design  is  of  importance  for  handling  this  problem.  Large  but 
small  diameter  containers  are  preferable  for  two  reasons;  first,  the  very 
low  resonant  frequency  of  liquid  in  large  diameter  containers,  and  second, 
the  larger  forces  toward  the  tank  wall  to  be  expected  in  large  diameter 
containers.  These  requirements  following  from  liquid  slosh  characteristics, 
however,  contradict  (in  most  of  the  practical  cases)  design  requirements 
which  suggest  containers  to  be  built  as  short  as  possible  in  order  to  avoid 
bending,  or  for  other  structural  reasons.  To  serve  both  requirements, 
longitudinally  compartmenting  the  large  diameter  containers  might  be 
considered  a  fair  approach  to  an  optimum.  Attempts  in  this  direction 
have  been  made  by  the  design  of  so-called  "scallop"  or  "multicell"  tanks. 
Even  though  the  first  resonant  frequency  of  the  compartments  will  not 
correctly  be  in  agreement  with  Equation  (2)  because  Equation  (2)  applies 
only  to  cylindrical  containers  (Ref.  1),  it  will  be  higher  than  in  the  single 
tank  and  thus,  be  more  advantageous  to  control  liquid  slosh  motion  as  dis¬ 
cussed  in  the  previous  section. 

The  cluster  principle  as  experienced  in  SATURN  I  may  be  considered 
an  incidental  modification  of  the  compartment  concept.  Here,  in  addition, 
the  effect  of  long  but  small  diameter  tanks  is  advanced  by  a  combination 
of  liquid  systems  of  different  resonances.  If  the  center  container  with  its 
lower  resonant  frequency  is  excited  to  the  extreme  splash  amplitudes,  the 
liquid  surfaces  of  the  outer  containers  are  still  smoothly  swinging  because 
the  frequency  ratio  A  in  the  center  container  is  high  (low  resonant  fre¬ 
quency)  while  the  frequency  raii.o  in  the  outboard  containers  is  relatively 
small  (higher  resonant  frequency).  This  combination  certainly  stablizes 
one  portion  of  the  available  liquid  propellants  while  the  other  portin''  is 
unstable  and  the  net  forces  on  the  tank  wall  are  reduced  accordingly. 

Equation  (2)  shows  that  the  resonant  frequencies  are  essentially 
dependent  on  the  n/d  ratio,  and  at  "shallow  water"  conditions  also  on  the 
h/ d  ratio  {/(.)•  This  means  that  the  first  resonance  frequency  is  very  low 
if  the  vehicle  acceleration  is  small  (n  — >  o),  which  concerns  upper  stages 
after  separation.  Since  such  stages  usually  are  designed  short  (large 
diameters)  for  structural  reasons,  the  ratio  n/d  and  thus,  the  natural 
frequency  of  the  liquid  system  is  extremely  small.  Any  vehicle  motion 
will  immediately  create  the  critical  splash  condition:  the  liquid  portions 
thrown  up  under  these  conditions,  but  dropping  back  to  the  liquid  surface 
very  slowly.  The  consequence  would  be  an  enhanced  heat  transfer  from  the 
low  temperature  liquid  to  the  higher  temperature  ullage  gas  which  could 
jeopardize  the  ullayr  pressure. 
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This  consideration  suggests  the  design  of  honeycomb  containers 
for  upper  stages,  the  cells  extending  in  longitudinal  direction  with  dia¬ 
meters  small  enough  to  provide  a  larger  n/d  ratio  (higher  resonance) 
for  each  cell  in  spite  of  the  low  vehicle  acceleration  (n).  Such  a  honey¬ 
comb  container  could  be  made  of  very  light-weight  (small  wall  thickness) 
because  the  pressures  toward  the  cell  walls  cancel  each  other  except 
on  the  outside  walls. 

It  should  be  noted  that  from  another  point  of  view  the  honeycomb 
concept  is  also  advantageous  for  solving  the  problems  of  liquid  behavior 
under  low  gravity  conditions.  It  is.  known  that,  at  low  g,  the  influence 
of  the  surface  tension  becomes  more  and  more  effective  on  the  shape  of 
the  liquid  surface.  J.  T.  Neu  and  R.  J.  Good,  in  an  interesting  study 
on  this  particular  subject  (Ref.  7),  also  arrive  at  the  conclusion  that  a 
honeycomb  container,  according  to  their  suggestion  of  conically  shaped 
hexagonal  cells,  would  be  the  proper  solution  for  controlling  liquid  pro¬ 
pellants  under  low  gravity. 

The  characteristics  discussed  so  far  provide  design  criteria  for 
preventing  violent  liquid  slosh  motion  on  a  natural  basis.  They  are  de¬ 
duced  form  the  existing  theory  which  is  in  good  agreement  with  the  exper¬ 
iment  for  the  most  critical  conditions  formulated  earlier  in  this  paper. 
Although  there  are  many  other  effects,  especially  during  vehicle  flight, 
which  can  change  the  liquid  motion  considerably  (interference  by  eng:  ne 
vibrations,  bending,  tank  breathing,  etc.  ),  the  conditions  discussed  above 
are  always  actual. 

In  the  past  the  procedure  almost  every  Lime  was  to  design  a  container 
which  satisfies  all  structural  and  weight  requirements,  and  to  consider 
liquid  slosh  motion  as  a  secondary  problem.  This  led  to  the  installation 
of  so-called  anti-slosh  or  slosh  suppressing  devices,  which  sometimes 
a:e  based  on  rather  eccentric  ideas.  Only  few  of  those  ideas  are  somehow 
reiated  to  theoretical  facts.  One  of  them  is  the  floating  can-lypc  anti- 
slosh  device  (Ref.  8)  which  has  been  successfully  flight  tested  some  ye.'rs 
a«jo  (Fig  5).  The  function  of  this  device  is  based  upon  the  pressure  aqua¬ 
tion  which  shows  that  only  the  upper  portion  of  the  liquid  down  to  a  depth 
of  about  one-quarter  diameter  of  the  container  is  in  motion,  which  deter¬ 
mines  the  length  of  the  cans,  and  based  on  the  fact  that  the  spring  constant 
of  the  liquid  system  in  motion  is  increased.  The  friction  between  the  fioats 
and  on  the  tank  wall  cannot  be  considered  responsible  for  the  damping 
characteristics  as  sometimes  erroneously  anticipated.  This  anticipation 
is  disproved  by  the  fact  that  can  devices  of  too  light  weight  where  the 
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friction  Li  supposed  to  be  ths  earn*,  are  not  feasible,  Tho  floating  bail 
which  cover*  th*  total  liquid  *urfac*  can  b«  considered  an  extended  can 
device,  Th*  floating  lid  which  also  covers  ths  total  liquid  surface  actually 
provides  for  all  liquid  levels  a  11  full  tank"  condition.  The  liquid  is  encap- 
suled  drainage  and  thus,  acts  like  a  solid  body,  The**  di vices,  of  ouurs*, 
are  feasibis  only  for  container*  which  are  long  compared  with  the  dia¬ 
meter, 

For  conceivable  reasons,  all  floating  devices  hava  many  opponent* 
particularly  among  designers,  JCmphdsis  Is  placed  on  eo-ealled  fixed 
devices,  Some  of  them  are  ihown  in  Figure  6,  In  most  easel,  these 
device*  ire  ring-like,  mounted  at  the  tank  wall.  Many  teste  proved  that 
simple  flat  rings  have  the  sain*  effect  ae  ths  other  mors  complicated 
device*  shown  In  Figure  6,  Th*  function  of  those  devices  actually. ha* 
no  correction  to  the  existing  theory  because  the  damping  eharaetnristic 
of  such  baffles  is  evidently  a  nonlinear  effect,  e  problem  which  le  not 
solved  yet  As  e  stop-gap,  a  linear  damping  factor  haa  besn  Introduced 
Into  th*  liquid  surface  aquation  In  a  similar  Way  as  customarily  Imple¬ 
mented  In  equations  for  mechanical  vibrations.  Ths  damping  factor 
then,  Is  determined  by  experiment,  It  has  bean  found  that  the  ratio  bet¬ 
ween  ths  ring  width  (w)  and  the  radius  of  the  tank  (r)  provides  a  damping 
factor  within  a  satisfactory  margin.  Th*  ratio  of  w/r  ■  0.  IS  may  be 
considered  e  practically  useful  average,  This  mean*  that  the  ring  width 
in  large  diameter  containers  is  considerable,  not  to  mention  the  necessary 
structural  strength  of  suoh  rings  because  they  have  to  consume  all  ths 
force*  exerted  by  th*  liquid  in  motion. 

The  solid  sruvs  U«vic»  -hewn  in  Figure  6  is  a  uompartmsntatlon 
of  th*  liquid  bulk  which  cl»*ng»-  the  natural  frequency  of  the  liquid  within 
ouch  compartment,  ae  discussed  earlier.  It  the  walls  of  ths  c«„j*  #r« 
pur, 'orated,  ths  ntfurei  frequency  of  ths  liqutd  will  not  change  aonslderabi, , 
but  ths  dumping  will  bo  satisfactory  (R->f,  9)  ,  All  th***  flxntl  d- vires 
except  compsrtmentstlnn  ars  comparable  with  "break-waters'1  in  the 
oeuan.  They  break  the  liquid  wave*  which  evolve  within  the  tank  accord¬ 
ing  to  the  theory.  From  this  point  of  visw  it  follows  again  that  no  •->.?- 
relation  between  ths  linearised  theory  and  the  bshavior  of  the  liquid  dus 
to  baffling  can  exist.  Th*  baffles  actually  disturb  what  the  theory  des¬ 
cribes. 
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In  recant  years,  another  effect  has  baan  applied  for  oontrolltng 
liquid  aloah  motion.  If  a  tuba  la  submerged  longitudinally  along  tha  Utnk 
wail  In  tha  plana  of  motion,  tha  phaaa  batwaan  tha  oscillating  liquid  sur- 
faeaa,  Inalda  and  outside  tha  tuba,  ohangqa,  If  tha  submerging  depth  of 
tha  tuba  oponlng  la  about  ona  tank  dlamatav,  tha  phaaa  shift  la  approxi- 
niataly  ISO  dagreaa,  Thta  maana,  at  thla  stage  the  liquid  laval  inalda  tha 
tuba  oaeillataa  Invsraaly  to  tha  laval  in  tha  container,  Tha  phaaa  ahlft 
takas  placa  while  tha  opening  of  tha  tuba  passes  tha  space  batwsan  a  dapth 
of  ons-quarter  and  three-quarters  of  tha  tank:  dtametar  below  the  aero 
liquid  laval.  At  a  constant  exciting  fraquanoy  tha  liquid  surface  In  tha 
tank  oaolllatas  with  amplitudes  according  to  aquation  (1),  while  the.  liquid 
level  inside  tha  tuba  la  going  up  and  down  aa  in  a  U- tub#  but  with  different 
phaaa.  Tha  sequence  it  illustrated  in  Figure  7.  Hare,  four  a  ami -annular 
tubas  are  attached  to  tha  container  wall,  ona  pair  in  the  plana  of  motion! 
the  other  pair  normal  to  the  direction  of  motion.  Tha  longitudinal  cross 
ssctlon  indicates  tha  differently  oscillating  lovals  in  the  tank  and  in  the 
tubas.  Tha  reason  for  the  phase  shift  has  not  boon  clarified  yet,  theoreti¬ 
cally.  Qualitatively  considered,  thla  configuration  roaamblaa  the  anti- 
rolling  tank  principle  which  haa  bean  appliad  to  large  ships  by  frahm, 
in  1902  (Raf.  10).  The  tubas  shown  In  figure  1  may  be  considered  the 
"absorber  system"  which  is  attached  to  tha  "ship"  (vehicle  container). 

In  contrast  to  Frahm'*  concept,  the  "eaa"  in  our  aase  is  Inside  tha  "ahip" 
and  excited  by  tha  "ahip"  motion.  Thia  principle  haa  been  successfully 
model-tested.  It  is  quite  different  from  tha  principles  applied  so  far, 
and  ahould  not  b«  confused  with  eompartmentiug  thu  tank  or  with  baffling. 
Practically  no  forces  due  to  liquid  motion  aro  acting  on  tha  tuba  walla 
Tha  preemre*  inside  and  outalda  tha  tuba*  cancel  nxuh  other.  Thua. 
the  tube*  can  b«  marts  of  very  thin,  lightweight  malarial,  By  thla  method, 
tha  liquid  ii  divided  in  at  ieaat  two  interfering  oscillating  ayatama  which 
do  not  dliturb  tha  coherence  of  iha  liquid  aa  baffle*  do  (vlule-1’  aolaahing) 
and  which  can  be  appliad  to  tha  moat  critical  ar*a*  of  tha  container, 

The  principle  illustrated  by  fig  ?  alio  can  bs  combined  with  a  iloal' 
mg  ball  device  aa  shown  in  fig  8.  The  double-walled  cylindrical  portion 
of  the  bell  ii  compartmrntad  so  that  each  compartment  acts  like  r  tube 
nr,  illustrated  in  figure  7.  Due  to  the  phase  shift  discussed  earlier,  the 
liquid  flow  in  tha  open  area  within  the  ball  and  in  tha  double-wall  section 
of  the  ball,  companaate  aach  othar  as  indicated  by  tha  arrows  in  tha 
•ketch  Figure  b. 
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LIQUID  SURFACE  MSASUP.INO  METHODS  DURING  SLOSH  MOTION. 
Another  problem  closely  eonnenTed'with  damping  liqu'li'eToeh  motion,  Ti. 
eh*  measurement  of  liquid  surface  deformation  particularly  during  vehicle 
flight,  The  method*  developed  and  utiliaed  *o  far,  however,  give  only 
poor,  and  even  milleading  information.  The  moat  common  method!  are 
differential  preeaure  and  capacitance  meaauremente,  Since  thee*  meas- 
uremente  are  only  point-measurements  of  the  liquid  surface,  a  compre- 
heneiv*  knowledge  on  the  real  shape  of  the  eurface  at  any  instant,  is  not 
possible, 

Thl*  experience  stimulated  a  study,  contracted  reee^tly  hy  MSEO 
to  SPACO,  INC,,  Huntsville,  onths  feasibility  of  the  atereo«iiel'evislon 
principle  for  monitoring  the  shape  of  the  total  liquid, eurface  during  *iosh 
motion,  This  study  cam*  up  with  the  interesting  Result  that  it  Will  b* 
possible  to  scan  the  liquid  eurface  with  a  TV-oam*ra  and  td  monitnr  and 
convert  the  data  into  a  contour  mapping  of  the  surface  at  any  instant, 

Such  contour  mapping  of  the  liquid  surface  characterised  by  contour  line* 
of  equal  liquid  level  would  provide  respectable  Information  on  liquld  bie* 
havior  under  varying  conditions  which  would  help  not  only  to  clarify  the 
frequency  spectrum  acting  on  the  liquid,  but  also  could  bo  utlUaed  a*  an 
improved  method  to  solve  the  problem  of  exact  propellant  loading. 
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fhe  general  equation  of  the  force*  toward  the  tank  wall  in  (Ref.  $ 
and  6)t 


For  the  amplitude  of  force  (a***  *  l)i  Wemgi  at  the  tank  wall  (rea)i 
K*tanh  2ann  t  and  the  firet  reeonance  of  liquid  («„« oi  wn«w4). 
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Substituting  in  0 )  and  transforming 
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Equation  (9)  lath*  basts  for  tht  nomograph  Tig  J|  X  U  a  *inetlon 
oj  d/xqK*  aocording  to  aquation  (4), 
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BASIC  CONSIDERATIONS  FOR  THE  PRELIMINARY  DESIGHf 
OF  A  SHOOK  TUBE  FOR  THE  INVESTIGATION  OF  THE 
ACTION  OF  A  NUCLEAR  EXPLOSION  WAVE  UPON  A  MISSILE 

Dietrich  E.  Gudaant* 

Structures  and  Meohlnlcs  Laboratory,  Army  Missile  Command 

Modarn  warfare  make*  it  naoaaaary  to  inoiuda  all  thoaaenvlrpnmentsl 
oondltlona  whioh  ara  expected  to  ba  imposed  on  all  typaa  of  mlaailaa  In  flight 
and  prior  to  launch  during  a  nud«ar  attack.  Tha  aaaanttal  affaota  of  a  nuclear 
explosion  ora: 

(1)  Tha  blest, 

(2)  Tha  haat  radiation,  and 

(3)  Tha  nuelaar  radiation . 

From  tho  point  of  viaw  of  tha  Structural  and  Meohar.ioa  Laboratory.  Army 
Militia  Oommand,  all  thraa  affacta  are  important.  Tha  bla.at  and  tha  haat 
radiation  endanger  tha  integrity  of  tha  misaila  atructura:  tha  nuclear  radiation 
aa  wail  aa  the  heat  radiation  oan  change  tha  properties  of  matarlala  to  an  un¬ 
bearable  extent  .  Little  la  known  aa  yat  of  tha  full'  affacta  of  shook  wave*  and 
tha  accompanying  extrema  tamparatura  gradlant.  tha  radiation  of  tha  fireball, 
and  the  nuclear  decay  on  mlaalle  atruoturaa.  In  order  to  meet  the  extrema  en¬ 
vironment  of  a  nuclear  explosion,  it  appaara  extremely  naoaaaary  to  Investigate 
the  fundamental  characteristic!  of  response  of  tha  missile  structure  under  suoh 
circumstances. 

Since  the  D'/namtu*  Analysis  Branch  of  the  Stri'clmes  and  Mechanios 
laboratory  1«  yiiinarlly  Interested  in  the  dynemioal  reaponae  of  mlaaila  struc¬ 
tures  to  a  nuclear  explosion,  this  paper  is  restricted  to  the  blast  offsets. 

The  problem  of  predicting  tha  charaatarlstloa  of  dynamic  loading  on  a 
iiKiille  structure ,  give rs  tha  peak  overpressure  end  the  duration  and  shape  of 
the  positive  end  negetlve  phases  of  the  blast  wave,  oan  ba  approeuhnn  in  four 
way*i: 


(1)  Theoretical  analysis, 

(2)  The  use  of  high  explosive  charges 

(3)  The  tuli-soale  nuclear  test,  and 

(4)  The  shook  tube. 


*The  author  is  now  at  the  NASA  Merehsll  Space  Flight  Center. 
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It  would  bs  like-carrying  coal  to  Newcastle  to  quaetton  the  uiafulnaaa 
of  tha  thaoratloal  analyala.  Tha  thaoratloal  analysis  ahould  dallvar  a  mathe¬ 
matical  modal  of  the  event*  when  a  bleat  wave  passes  over  a  mlaatla  end 
■hould  alio  develop  a  modal  for  tha  atruotura  mathematically,  bated  on  struo- 
tural  raaponaa  oharaotcrlatloa.  However,  the  analytical  method*  are  not  yet 
developed  enough  to  relinquish  experimental  Investigation*.  Among  the  ex¬ 
perimental  method*  the  shook  tube  la  admirably  aultad  to  studies  of  Utla  nature. 

It  provide*  a  laboratory  controlled  safe  method  for  obtaining  shook  wave*  of 
dealred  peak  praaaurea  In  a  relatively  Inexpenalve  manner)  theae  ahook  wave* 
may  be  impoaad  on  structural  models  of  arbitrary  ahape a,  and  of  courts,  of 
limited  dlmenaiona.  Accurate  preaaure-tlma  meaaurementa  can  be  made  on 
aaoh  taoe  of  the  missile  model.  Naturally,  the  ahook  tuba  haa  not  only  advan¬ 
tages  but  also  a  few  disadvantages  euch  aa  limited  control  of  the  ahape  of  the 
pressure  pulse  following  tha  Initial  pressure,  and  a  limited  duration  of  the 
pressure  pulse.  However,  despite  these  small  disadvantages,  the  shook  tube 
has  been  In  tha  past  an  extramely  powerful  tool  for  the  design  of  blast-resistant 
protective  atruotursa  and  should  also  usefully  serve  In  the  future  for  the  Inves¬ 
tigation  of  the  effects  of  shock  waves  on  missile  structures. 

Please  let  me  give  you  first  e  brief  survey  of  the  phenomena  after  a 
nuclear  explosion  In  tha  atmosphere.  Almost  Immediately  aftar  the  detonation 
oaour*.,  the  expansion  of  the  hot  gases  Initiates  a  pressure  wave  In  the  aur- 
rou,.ding  air.  As  the  pressure  wave  propagates  away  form  the  center  of  the  ex¬ 
plosion,  the  following  (or  Inner)  part  moves  through  a  region  which  has  been 
previously  compressed  and  heated  by  the  leading  (or  outer)  parts  of  tha  wave. 

The  diciarbance  moves  with  the  velocity  of  sound  and  since  this  velocity  In¬ 
creased  with  temperature  and  oreseure  of  the  air  through  whloh  tha  wave  Is 
moving,  the  Inner  pert,  of  tha  wove  moves  mors  rapidly  and  catches  up  with 
the  outer  part.  Tha  wavefront  thus  gats  deeper  and  ateepor  and  within  a  vary 
short  period  It  bsoomss  abrupt.  Tha  advancing  shock  front  provldas  a  vu.y 
sudden  Inoraasa  of  praasura  from  normal  atmoaphatlo  to  the  peek  shbok  pleasure. 
This  shook  front  thus  behaves  like  a  moving  well  of  highly  compressed  el* . 

As  ths  expansion  proceeds,  the  pressure  diet) .button  In  the  region 
behind  the  shook  front  gradually  change*.  The  overpreeeure  la  no  long*,  con¬ 
stant  but  drops  off  continuously  toward  the  center.  At  later  time*  when  the 
shock  front  has  progressed  some  distance  from  the  center,  *  rarefaction  wave 
develops  at  the  canter  causing  a  drop  in  pressure  below  the  Initial  etmoapharlo 
value,  Thus,  a  suction  phase  develops.  The  shook  wave  weakens  ao  It  pro¬ 
gresses  outward  end  Its  voloolty  drops  toward  the  velocity  of  sound  in  the  In¬ 
itial  cooler  air. 
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If  the  bomb  is  detoneted  at  •  distance  h  above  the  surface  of  the  earth, 
the  shook  wavs  will  have  tha  general  configuration  indicated  in  Figure  1  (Figures 
are  at  the  end  on  this  article)  during  the  brief  time  interval  before  It  Impinges 
upon  the  surface , 


A  short  time  later  the  radius  of  the  shook  front  becomes  greater  than  n  , 
and  that  protlon  of  the  incident  shock  wave  which  impinges  upon  the  earth's 
surfaoe  is  reflected  back  forming  the  reflected  shook  wave  as  illustrated  in 
Figure  2,  The  symbol  <*•  represents  the  angle  of  Incidence  of  the  shook  wave 
with  the  earth's  surfaoe.  The  reflected  shook  wave  overpressure  Pr  is  a 
function  of  tha  Incident  shook  overpressure  Pg  and  the  angle  of  Incidence  , 
but  always 

P,  2  P,  . 

The  raflactad  shock  front  travels  through  the  atmosphere  at  a  higher 
velocity  than  the  incident  shook  end  gradually' overtake*  and  merges  with  it 
to  form  a  single  shook  front  called  the  Mach  stem,  as  shown  in  Figure  2.  The 
fused  shook  front  thus  formed  is  normal  to  and  travels  parallel  to  the  earth's 
surfaoe.  l'ne  Mauh  slum  formation  is  initiated  when  tho  angle  of  Incidence  of 
shook  wave  becomes  greeter  than  approximately  46°.  Once  formed,  the  hklght 
of  the  Mach  item  gradually  Increases  es  the  radius  of  tha  shook  wave  beoomue 
greater. 

The  Importance  of  the  Mach  stem  phenomena  Is  that  It  causes  two 
».L3ck  waves  to  fuse  Into  s  single  shock  wavs  of  higher  overpressure  anu  of 
greater  destructive  powe,  to  atructures  locetod  In  Its  pat!.. 

The  peak  overpressure  P„  existing  In  tho  shook  wav  a  edlaosnt  to  the 
ground  surfacs  is  a  function  of  the  distance  from  the  point  of  burst  and 
yt«ld  of  the  weapon;  its  value  Is  plottsd  In  Figure  3  for  four  weapon  *Uea, 

20  KT,  100  KT,  1  MT,  and  20  MT..  These  curves  are  weapon  bums  at  ground 
surface.  Tht  shook  front  velocity  U  is  a  faction  of  the  peak  overpressure 
P(J  (Figure  4).  Its  value  is  given  through  the  following  equation! 


U 


cn< 


i+j.  _li_ }  1/2 

2  2  P0  1 


whore 
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Cq  *  ambient  speed  of  sound 
S’  “'ratio  of  tha  apaoiflo  hast  ef  air 
Pq  ■  ambient  praasura  (ahead  of  tha  tihock  front)  of  tha  atmoaphara. 

With 


JL 

°0 


M, 


(Shook  Mao!  i  Number) 


the  equation  almpllfiea  to 


.  7  P0 


1/2 


where  tha  velocity  of  the  shook  front  La  eXpieeeed  ae  a  multiple  of  tha  amblant 
sound  speed, 

After  a  reflection,  wha  shock  wave  becomes  stronger  and  the, reflected 
ovarpraaaura  ratio,  t^P,  ,  la  plotted  in  Figure  S  aa  a  1’unotlon  of  Angle  of 
tncMHvir.oa,  o|  ,  of  tho  ahook  front,  This  figure  applies  to  both  an  inclined 
ahook  front  striking  tha  surface  of  tha  earth  and  a  ahook  hunt  striking  any  sun- 
face  an  angle  of  Incidence  . 


In  the  second  part  of  my  pepar  I  want  to  dkeouee  tha  deeign  orlterta  of 
a  shoe)-  tube  in  order  to  simulate  a  nuclear  explosion,  Pleaue,  let  me  give  you 
first  a  brief  survey  of  th«  elementary  enook  tube  theory. 


figure  6  shows  the  most  simple  snook  tube  assembly.  The  shook  tube 
consists  of  a  rigid  oyllnder  divided  Into  two  sections  by  e  gastlght  diephuirr 
Gne-huK  of  the  tube,  known  as  the  compression  chamber,  contains  a  gas  at  a 
pressure  p4  ,  which  is  in  excess  of  the  pressure  p,  of  the  gee  In  the  other 
half  of  tho  tube  known  as  the  expansion  chamber,  Tno  gases  on  etthur  side 
of  the  cilaehragm  need  not  necessarily  be  of  the  tame  chemical  type 

Whtn  the  diaphragm  is  caused  to  shatter,  a  shock  wave  travels  Into 
the  expansion  chamber  and  a  rarefaction  wave  travels  back  Into  tha  oompreaeion 
ohamh.r.  A  flow  of  gas  is  behind  the  shook  front  and,  In  order  to  avoid  pressure 
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variation*  building  up,  tha  flow  velocity  la  uniform  In  the  rogion  batwaan  tha 
shook  front  and  tha  tall  of  tha  rarefaction  wave.  This  1*  also  a  region  of  oon- 
atant  pressure.  Tha  distribution  of  pressure  along  tha  tube  before  and  aftor 
tha  diaphragm  has  shattarad  Is  also  shown  on  tha  ploture. 

The  dotted  linn  denote*  the  position  oooupled  by  that  gas  which  was 
originally  at  the  diaphragm.  Tha  gas  to  tha  right  has  baan  compressed  and 
heated  by  tha  shook  wave  but  tha  gas  td  the  left  of  this  Una  has  expanded  from 
tha  compression  chamber  and  has,  therefore,  bean  cooled.  At  this  position 
there  will  be,  therefore,  in  general,  a  ohanga  of  type,  temperature,  anddenaityi 
vnioolttoe  are  the  same  on  both  side*.  Such  a  point  la  known  aa  a  contact 
dlecbntlnulty.  ... 

Figure  7  shows  a  plot  In  tha  K,  t  plana  (x  corresponds  to  tha  length 
of  me  shook  tuba  with,  x  ■  0  at.  th*  dlaphiagm  and  t  la  tha  time)  of  tha  pro- 
oosses  occurring  In  a  shook  tuba  with  both  and*  closed.  Tha  shook  front  posi¬ 
tion  la  represented  by  the  line  OA  with  elope  dt/dx  «  l/U  (U  '*  velocity 
of  th*  shock  wave)  and  tha  contact  discontinuity  by  OB  with  alopa  dx/dt  •  l/u 
(U ’■  velocity  of  tha  'contact  discontinuity),  This  meats  tha  reflected  shook 
AS  at  B  .•  where  tha  shook  wavs  once  ag*ln  undsrgoas  •  ref  faction  (not  shown 
in  the  plot),  Ths  slope  of  the  contact  aurfaoe  curve  In  the  x,  t  plana  Is  than 
very  steep  because  the  flow  velocity  la  low  In  this  region.  Tha  head  of  th* 
rerafaotlon  wave  travelling  frum  tha  diaphragm  la  reprorentad  by  tha  character¬ 
istic*  0  0  and  th*  path  of  the  reflected  rarefaction  wave  la  denoted  bv  DE 

Figure  8  show*  tha  sltuat  on  In  a  ahock  tuba  a  short  tlms  after  the  dia¬ 
phragm  wa*  removed.  The  strong  line  r  .vas  tha  pressure  distribution  vareua 
distance,  ths  da3h.au  line  the  original  pressure  dtstiloutlon,  and  the  daahad- 
dottod  line  Ui*  contact  surf*r,e. 


coroaa 

frent; 


Finally,  ths  next  equation  will  yield  the  static  pressure  ratio 

the  diaphragm  necessary  to  give  a  pressure  ratio  actors  the  ohock 


1  -  A ,  A 

(  -  1)  (1  “  Pjjj) 

A4 

f  2^1  pia[p22(y  -  D+  _ 
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For  V  A  -  7/5  end  AJ4  “  aj/e4  »  1  follows 


pH“  pl2 


*-pl2  1  7 

T~P12(pl2  +  6>?  1/2  J  ' 


It  la  olatr  from  the  first  equation  thet  the  starting  preaaurc  ratio  P, . 
required  to  prpdue*  a  shook  of  pressure  ratio  la  a  function  of  three  In**' 
dependent  parameters ,  A.,  ,  V,  and  <jf  4  .  A,4  la  the  ratio  of  the  ' 

speeds  of  sound  in  the  region  1  and  4r  V,  ,  V.  is  the  specific  heat  In 
region  1  end  4.  The  theory  may,  of  course,  be  ftrmulated  lh  terms;  of  bther 
parameters,  suoh  as  internal  energy,  for  example.  The  primary  oondltlon  for  1  , 
producing  •  shook  Is  the  dtfferanoe  In  pressure  acrosp  the  diaphragm  and,  . 
having  fixed  j  1  and  V.  by  a  ahoioe  of  gases,  one  still  has  a  free, para¬ 
meter  which  becomes  fixed  when  the  temperature  ratio  Is  epaotflsd.  In  prac¬ 
tice  it  is  found  very  advantageous  to  use  a  light  gas  tp  drive  a  heavy  one, 
suoh  as  helium  on  air,  because  this  combination  gives  a  smaller  value  of  Ai:i. 
without  change  In  temperature,  and  results  In  a  stronger  snook  without  In¬ 
creasing  the  pressure  ratio  eoross  the  diaphragm,. 


Figure  9  shows  the  relation  between  the  shook  Much  number  Ma  ehd 
the  pressure  ratio  P42  tomes  the  diaphragm  <« 1  several  values  W  • 

For  s  strong  shook  the  following  squat! on  yields  the  shock  Mach  number) 


M 


s 


r 

1 


Vj  +  1  ♦  (  ^  a  l)P12 
"  - 


1/2 


Using  a  nerloln  substitution  yields 


2 

M, 


V, 


+  1 


2ll  * 


-  iv 


Figure  10  shows  the  variation  of  the  shook  pressure  ratio,  ,  with  the 
diaphragm  pressure  tetlo,  P41  ,  end  Figure  11  the  variation  of  the  snook  Meoh 
number,  Mi  .  and  pirticln  velootty  uj  ,  with  diaphragm  pressure  ratio,  P4, 
for  the  gas  combination  alr/alr,  He/etr,  or.d  Hj/  sir. 
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Before  any  deelgn  data  oan  ba  givan,  It  la  naoaaaary  to  dbclds  which 
apaolflo  condition!  shall  ba  simulated.  Any  point  around  a  nuclear  axploalon 
la  ohavaotarlaad  by  ovarpraasura  and  duration  of  flow.  Hence,  It  muat  be  deter¬ 
mined  for  what  ovarpraasura  and  flow  duration  tha  shock  tube  shall  ba  dealannd, 
though  tha  flow  duration  la  of  minor  importance  since  (pedal  instrumentation 
and  theoretical  considerations  elmlnete  the  need  to  simulate  the  full  duration 
of  flow. 

Although  present  missiles  oan  only  ondure  a  very  small  overpressure, 
it  appears  reasonable  that  future  missiles  should  bear  a  much  hlghar  over-  - 
pressure  in  tha  magnltuds  of  190  psl.  100  pal  is  almost  equivalent  to  a  pressure 
ratio  of  tha  shock  wavs  of  It  8. 

Formerly  given  graphs  hava  shown  Immediately  that  the  desired  pressure 
ratio  of  the  shook  front  P2l  "  8  i*  obtained  by  a  pressure  ratio  aoross  the 
diaphragm 

•  200  for  eir/alr 

P4l  -  26  for  Ha/alr 

P4l  -  16  for  H?/alr, 

Tno  notation  alr/alr,  He/alr,  and  H2/alr  means  that  sir,  helium,  and  i.ydrogen 
are  used  as  driver  gasoe  In  the  high4 pressure  chamber,  respectively,  and  only 
...r  Is  used  as  driven  gee  in  the  low  pressure  chamber. 

The  pressure  retro  t',,  *8  yields  a  velocity  ratio  of  tha  ahook  wave 
M  «  2,6.  If  the  ahoak  tuba  is  designed  to  bear  an  overpressure  of  about  3000 
p«i  (P4,  -  200),  then  it  is  possible  to  attain  shock  pressure  retlo  **21  /' 

With  helium  ana  P2i  “  23  with  hydrogen  Instead  of  all  aa  driver  oas  which 
is  equivalent  to  about  *30  psl  and  34S  psi,  respectively . 

Tha  numbers  givan  above  show  oleerly  thet  la  is  easily  posetble  ,o 
simulate  the  psak  ovarprasiurs  of  a  nuclear  blast  with  a  shook  tuba  buill'with 
moderately  strong  material. 

Tha  length  of  tha  shook  tubs  la  a  vary  important  faotor  since  tha  dura¬ 
tion  of  the  flow  depends  on  it.  Figure  12  represents  the  course  of  events  by  a 
ttma-dlaience  plot  which  can  ba  uaad  to  determine  the  length  of  a  shock  tuba. 

In  the  picture  the  origin  Is  tsken  at  the  position  of  the  diaphragm  and  al  the  tlmu 
it  breaks.  The  abscissa  X  •  x/%  Is  distance  along  the  channel  divided  by  the 
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length  of  th«  praatura  chamber  $  ,  The  ordinate  V  *•  a,  t A  Is  likewise 

dimensionless  and,  hence,  the  plot  la  applicable  to  any  shook  tuba,  The 
velocity  of  the  shook  front  is  considered  as  constant  and  has  baen  previously 
presented  in  the  equations.  Its  path  Is  represented  by  the  straight  Una  a. 

The  contact  surface  La  represented  by  the  straight  Uno  b.  The  rarefaction 
wave  la  designated  by  d  and  after  reflection  on  the  closed  and  by  a. 

At  any  particular  X  the  length  of  a  verloal  Una  within  a  sons  whloh 
It  crosses  represents  tho  duration  of  the  flow  In  that  sons  for  an  Ideal  shock 
tube ,  The  verloal  line  through  the  point  B  extending  from  point  B  to  the 
intersection  with  the  line  a  at  B1  represents  the  longest  possible  time  of 
flow  between  the  shook  front  and  the  arrival  of  the  contact  surface  before 
other  disturbances  given  by  the  rarefaction  wave  may  ooour, 

,  In  ordar  to  oaloulata  tha  vary  Important  point  B  for  any  glvan  paramatera 
•4,  a^,  and  84  tha  following  equations  can  be  used: 


•>'  4  u2 


JJi.il  ♦  A 
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The  following  equation  yield*  the  length  L  of  the  oompreealon  chamber  *x- 
prened  in  the  asm*  system  of  unit*  in  which  aj  it  given: 


X  - 


ai rM. 

YB  M.-*B 


Consequently,  the  overall  length  L  of  the  whole  ahoclt  tube  la  given  by: 


L  *  JXB  ♦  l 


Aiaumtng  a  praaiura  ratio  *  0,15,  a  flow  duration  ?■  3  sec,  and  a 
velocity  of  aound  in  air  a,  ■  932  ni/s*o,  the  equation*  yield  the  overall  length 
of  a  rhook  tube  to  S,  2  kilometer* .  Stronger  shook*  aa  aaaumed  or  uaa  of 
helium  or  hydrogen  would  lead  to  even  longer  tube*.  From  the**  results  it 
beooma*  obvloua  that  it  la  practically  lmpoaalble  to  aimulata  the  full  duration 
of  the  ovarpreaaur*  phaae,  but  a*  already  mentioned,  the  need  tor  thla  can  ba 
ellmlnaied  by  apeclal  arrangement*. 

If  the  equation*  are  aolved  for  T  ,  the  flow  duration  behind  the 
nhook  wav*  In  a  ahook  tube  of  given  length  of  the  eompreaalon  chambe’  oan 
computed. 

Assuming  aqaln  i*  -  0.15,  aj  ■  332  rr/aeo,  and  the  length  L»  30  m, 
tho  oquatlou  yield*  t*-  U .  (fie  aeoonda. 

No  principal  consideration*  oan  be  given  for  the  diameter  or  the  ehook 
i-ilie  since  no  primary  physical  factor*  depend  on  thle  parameter.  Nevert.hete.ta, 
in  order  to  .'^vold  boundary  layer  effect*  and  aa  far  .aa  possible  to  eeen  the 
measurement*,  tho  dlemeter  of  the  shock  tube  should  bo  as  large  an  Justifiable 
from  the  atendpoint  of  general  design  and  economics. 

It  has  been  shown  that  from  tha  environment  of  a  nuclear  explosion  the 
most  Important  part,  namely  the  blast  wave,  oan  be  simulated  with  a  shock 
tube.  Other  parameters  aa  flow  duration  and  tha  shape  of  the  posltlvo  phase 
of  the  overpressure  are  difficult  to  simulate,  yet  the  need  for  thle  oan  be 
eliminated  by  apeolal  Instrumentation  and  theoretical  considerations. 
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In  order  to  meet  future  requirement;*  of  ml  title  structure!  An  re*  peat  to 
blavt  Loading  It  is  necessary  to  produoe  a  prenure  ratio  aero**  the  shook 
front  P2i  ■  8  which  Is  equivalent  to  an  overpressure  p$  “  108  psl.  The  simple 
:hook  tube  theory  shows  that  this  requirement  can  easily  be  achieved  by  a 
pressure  ratio  P,,  ■  200  (alr/alr)  across  the  diaphragm  of  the  shook  tube. 
Length  and  dlamnar  of  the  uhook  tube  should  be  as  large  as  justifiable  from 
the  standpoint  of  general  design  and  economics. 
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INCIDENT 


HOURE  i  SHOCK  FRONT  MOM  AEOVE-OROUNO  NUCLEAR  EXPLOSION  REFORE 
AND  AFTER  REFLECTION  ON  THE  SURFACE  OF  THE  EARTH; 


FIOURf  2  SHOCK-REFLECTION  PHENOMENA  IN  RE4I0N 
INHERE  a  IS  SREATER  THAN  4»*. 


DISTANCE  PNOM  MOUND  IERO,<YAAOFJ 

PlftUNI  3  PEAK  OVERPRESSURE  VSRSUS  DISTANCE  PON  SEVERAL 
YIELD!  OP  NUCLEAR  NOMNI. 


PEAK  OVERPRESSURE,  P((PII) 

PIOURP  4  »HOf.K- FRONT  VELOCITY  VS  PEAK  OVERPRESSURE 


0*  10*  10*  JO*  40*  80*  *0*  TO*  10*  *0* 


ANQU  Of  INCIOCNCt,  • 

flSURt  S  RifLSCTIO  OVIRfRUSURt  RATIO  V«  ANOLI  Of  INOIDCNCC 
fOR  VARIOUS  RKAK  OVKRfRCISURtt. 


FIGURE  10  VARIATION  OF  SHOCK  PRESSURE  RATIO,  Ptt 
WITH  DIAPHRAGM  PRESSURE  RATIO 


RELIABILITY  CONCEPTS  POR  MISSILE  BATTERIES 
Nicholas  T.  Wllbwrtj 

U.  B.  Army  Elactronloi  Research  and  Development  LeiioMbry 
Fort ' Monmouth,  Nfw  Jersey  \ 


This  lanlon  presents  a  long-awaited  opportunity  for  a  dleouseioi,  of  , 
mathodi  of  determining  th«  reliability  of  one-ihot  battery  power  supplies  for 
mtaallea .  Approaohei  and  suggeitlone  are  urgently  required  a*  to  appropri¬ 
ate  tost  designs:  the  nature,  extent  and  general  advisability  of  making  re-: 
liability  predictions  based  on  analysis  of  limited  amounts  of  test  data;  andV 
the  tiimsequuiuwa  of  u»«umpUoiii  modii  tn  regard  to  dlotrlbutlonn  and  vari¬ 
ance  of  the  battery  responses. 

The  betteiles  we  ere  dealing  with  are  ona-ehot  types.  Installed  In  A 
missile  to  'urnlsh  power  to  quldanoe,  oontrol,  or  werhead  systems.  Thsy  - 
are  star  .  m  an  inart  condition  in  the  missile  end,  upon  command,  muet  be 
activated  almost  Instantaneously  to  provide  power.  Due  to  the  inert  atate 
prior  to  activation,  there  la  no  way  to  taet  the  operational  raadlneea  of  an 
Individual  battary.  Tha  naoaeaary  reliability  muet  be  anglnaared  Into  the 
Uttery  during  Its  development  and  maintained  In  production  by  adequate 
quality  assurance  technique*. 

The  problem  areas  In  making  reliability  predictions  may  bast  be  under- 
B'  .od  by  first  considering  a  typical  mlselle  battery  end  Its  electrical  atiu 
environmental  requirements .  Electrical  requirements  aro  illustrated  In 
Figure  1  (Figure;  are  at  the  end  of  the  article;. 

The  shaded  area  la  the  time- voltage  envelope  which  defines  tnti 
o'ectrlcal  requirements.  Following  the  activation  Impulse  at  time  *ero,  the 
battery  voltage  must  rise  to  the  minimum  under  load  by  the  end  cf  the  actl- 
vrfilon  period,  generally  one  second  or  less.  The  voltage  under  nominal 
resistance  and  under  high  or  low  resistance  puleee  must  then  remain  net 
wuen  the  minimum  and  maximum  throughout  the  required  service  life  through¬ 
out  the  specified  temperature  range  and  while  being  subjected  to  dynamic 
environments  such  as  shook,  vibration,  acceleration,  and  spin.  Tha 
battery  design  should  of  ooursu  be  Invulnerable  to  environments  such  as 
temperature,  humidity,  and  transportation  and  handling  stresses  during 
long  storage  prior  to  activation. 
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Assuming  that  a  battery  .has  been  developed  lor  a  apeulflo  appltoa- 
tlon  and  that  there  is  reasonable  assurance,  based  on  extensive  testing 
during  development,  that  the  battery  is  capable  of  meeting  all  operating 
level  clecttlcal  requirements  while  being  subjected  to  the  specified  thermal 
and  dynamic  environments,  the  queetton  is,  how  can  tha  battary  reliability 
ba  tested  for  and  thus  assured?  By  reliability  we  mean  a  high  probability  of 
the  battery  activating  properly  upon  command  and  then,  fulfilling  Ita  mission 
which  is,  basically,  meeting  all  specified  requirements  under  any  and  all 
specified  environmental  stresses.  The  requirements  am  assumed  to  have 
been  accurately  stated. 

Figure  2  lists  some  of  the  major  problam  areas.  First  ara  the  relia¬ 
bility  standard!  whloh  must  ba  eatabllshtd,  Theaa  must,  to  be  practical, 
represent  some  compromise  bstween  the  desires  of  ths  missile  people  and 
the  battery  state  of  the  art.  A  reliability  of  99.999%  at  a  confidence  level 
of  99%  may  ba  desirable  whsrees  a  standard  of  99,9%  with  9S%  confidence 
may  be  an  aohlavabla  goal. 

Onoe  the  reliability  standards  have  been  determlnad,  the  dealgn 
qualification  procedures  must  be  established.  An  adequate  reliability  tost- 
lug  .-onoept  Is  tha  major  oonstdaration.  This  will  bs  discussed  later  et  some 
length.  The  capabilities  of  test  equipment  must  be  considered,  how  well 
they  will  duplicate  the  missile  environments.  Test  equipment  whloh  will 
simultaneously  simulate  moro  than  one  dynamic  environment  Is  normally  not 
available.  The  single  test  concopt  Is  Intriguing  In  this  respect,  an  artificial 
environment  whloh  will  aontaln  the  resultant  forces  of  all  of  the  major  dynamic 
foraea--shock,  vibration#  auooleretton,  eto.  Unfortunately  this  to  Just  a  i 
thought  at  present.  And  finally,  what  Is  the  nature  of  the  statistical  distri¬ 
bution  of  the  battery  responses  to  the  environments?  The  Bample  tes*  'emits 
will  U*  used  to  make  reliability  predictions  based  on  the  response  distribution* 
whloh  rtra  generally  unknown,  the  nature  of  which  must  themselves  Lo  deduced 
from  the  sample  results. 

Finally  when  tha  design  has  been  qualified  and  is  ready  for  production, 
thero  era  many  Important  considerations  to  assure  that  no  potentially  defective 
units  reauh  llie  field,  These  ere,  however,  beyond  the  scope  of  this  paper. 

The  major  concern  at  present  Is  the  design  qualification,  testing  a 
sample  of  batteries  to  determine  If  the  reliability  Ik  adequate  In  order  to  go 
Into  production  and  field  use,  or  If  additional  development  U  required  to 
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obtain  the  desired  reliability.  At  this  point  I  would  like  to  describe  briefly 
the  various  approaches  that  have  been  proposed  and,  in  some  instances, 
used  by  the  military  to  determine  "reliability" .  These  are  outlined  in 
Figure  3 . 

The  first  method,  testing  at  the  "four  times  operating  level",  Is  a 
non- statistical  approach  which  evolved  as  an  early  alternative  to  test-to- 
fallure  and  which  has  been  applied  with  considerable  success,  It  Involves 
testing  a  small  sample  (10  to  20)  at  four  ttmas  the  epeolfled  g  level  of  a 
dynamic  environment  (shock,  vibration,’ etc.),  The  method  searches  out 
mechanical  weaknesses  which  can  be  corrected  through  redesign.  The  four 
times  factor  Is  arbitrary.  It  was  felt  that  batteries  oapebls  of  meeting  this 
requirement  would  certainly  have  a  high  probability  of  meeting  the  spaolfled 
g  forces.  Only  one  dynamlo  snvlronmsnt  la  applied  at  a  time.  The  total 
sampla  slue  depends  upon  the  number  of  different  orltloal  environmental 
oondltlonai  high  temparatura  shook,  low  temparatura  shook,  high  tempera- 
ture  vibration,  etc.  Temperature  safety  faotora  are  Investigated  by  tasting 
beyond  the  epeolfled  temperature  range  In  the  unaotlvatad  condition.  One 
serious  disadvantage  of  this  method  Is  the  Inability  to  make  a  statistical 
prediction  of  the  probable  failure  rata  of  the  battery.  Reliability  muet  be 
expressed  somewhat  vaguely  a*  "high,  engineering  confidence  In  the  design" . 
Secondly  it  mey  be  argued  that  If  adequate  safety  factors  have  been  consi¬ 
dered  In  establishing  the  environmental  requirement* ,  then  forcing  the 
battery  design  to  operate  at  higher  stress  levels  may  unnecessarily  strengthen 
the  design,  possibly  increasing  cost,  weight  and  sUe,  Thirdly,  testing 
oi.ly  one  dynamic  environment  at  a  time  overlooks  potentially  destructive 
fo1  res  resulting  from  combinations  of  environments  which  are  actually 
experienced. 

The  aeoond  method,  presently  being  applied  In  the  development  of  a 
mluslle  battery,  contains  all  of  the  elements  of  the  first  plus  an  analysi»  c* 
the  variance  of  performance  parameters  such  as  service  life,  activation  time, 
maximum  vnltngn,  minimum  voltage  under  pulsus,  otc.  It  is  assumed  that 
the ee  responses  ore  normally  distributed.  Therefore  a  standard  may  be 
established  for  the  number  of  standerd  deviation  units  which  a  small  tost 
sample  (IS  to  2 5)  muet  demonstrate  between  the  sample  mean  end  the  operating 
requirement  level  for  each  performance  parameter,  This  number,  the  k 
faetbr,  Ib  related  to  sample  site,  the  desired  maximum  failure  rata,  end  the 
confidence  level  used  in  making  the  reliability  prediction.  The  total  program 
thus  gives  a  high  engineering  confidence  in  the  battery  design  from  a  mech¬ 
anical  standpoint  end  also  a  high  statistical  confidence  In  the  battery  design 
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meeting  that  electrical  operating  requirements  while  being  subjected  to  each 
environment.  Each 'dynamic  environment  is  studied  separately  et  either  high 
or  low  temperature  and  at  tour  times  the  specified  g  level.  The  possible 
disadvantages  of  the  method  are  those  as  stated  for  the  previous  method 
plus  the  fact  that  an  erroneous  assumption  of  normality  may  weaken  the 
value  of  the  reliability  predtctiona. 

1  The  thlrdimethbd  is  similar  to  the  one  juat  described  sxcopt  that  in¬ 
stead  of  using  an  arbitrary  multiple  such  as  fqur.  the  specified  g  forces  for 
the  dynamic  environment#  are  applied,  the  method  pa#  the  same  potential 
disadvantages  as  the  pravlous  one  in  overlooking  possible  Interactions  bet- 
waon  forces  and  In  assuming  normality  of  response  distributions. 

Ths  fourth  concept  approaches  the  problem  of  reliability  by  determining 
the  actual  mean  failure  points. of  ths  buttery  dsslgn  with  respect  to  saoh 
environment,  thermal  or  dynamic >  determining  the  variability  around  the  mean 
failure  point  and  making  a  pradictibn, of  the  battery  capability  at  the  require¬ 
ment  Itoyel  fbr  the  Individual  environment.  A  method  of  this  type  .was  dis¬ 
cussed  by  Mr,  H.  J.  Unglis  of  the  Ford  Motor  Company  at  the  eighth  con¬ 
ference  at  WRA1R  lest  year  and' was  the  subject  of  much  of  Profeaser  Ohernoffa 
tnvi»ad  talk.  In  this  approach  the  individual  environmental  teat  oondltlona 
«re  investigated  In  sequence.  Thus  interactions  between  environmental 
stresses  are  not  considered.  The  assumption  of  normality  Is  mada  In  ragard 
to  the  distribution  of  fallura  points  In  ordar  lor  predictions  to  be  made  with 
small  uamplas  In  the  order  of  IS.  Theunethod  can  be  applied  only  If  the 
teat  oi’ul.pmant  range  is  capable  of  inducing  failure*.  This  has  proved  to 
be  a  problem  In  many  lo«t.».ncc3. 

The  last  method  which  has  been  proposed  is  again  based  upon  deter¬ 
mining  mean  failure  points  and,  based  on  analysts  of  uartanoo,  predtotinv 
the  picnablllty  ot  fallura  at  tha  raqulramant  atraas  lavala.  Tha  method 
visualises  tha  detrimental  affsots  of  lnoreaalngiy  higher  environmental  sires* 
levels  on  ths  battery  performance  response*  such  that  each  performance 
parameter  may  ba  expressed  as  a  function  of  the  environment,  thermal  or 
dynamic,  or  as  a  function  of  as  many  environmental  variable*  as  may  bu 
studies  simultaneously  in  accordance  with  tho  capability  ot  test  equipment. 
Tolerance  Intervals  about  the  response  surfaces  thus  generated  can  then  be 
used  for  predicting  the  probability  of  failure  at  the  requirement  levels  or  at 
any  settings  of  the  variable*  throughout  the  test  region.  Actual  failure  need 
not  be  Induced.  The  problems  with  this  method  are,  again,  making  assumptions 
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of  normality  and  of  uniform  variance  throughout  the  environmental  test  regions 
as  well  as  the  difficult  problem  of  developing  statistical  techniques  for  ob¬ 
taining  multt-variate  tolerance  surfaces  and  being  able  to  rely  on  predictions 
made  from  the  testing  of  relatively  small  samples. 

This  general  background  of  the  attempts  that  have  bean  made  to  deter¬ 
mine  the  reliability  of  one-shot  Items  should  serve  lo  Illustrate  some  of  the 
problems  confronting  us.  The  goal,  basically,  Is  to  develop  techniques  with 
which  we  can  comp  up  with  meaningful  reliability  prediction  numbers  from 
the  testing  of  one-shot  items  where  the  sample  sices  are  necessarily  small 
dti?  to  the  relatively  high  cost  of  the  Individual  Items.  An  Important  question 
in  determine  our  approach  Is,  If  we  can’t  duplicate  the  specified  missile 
environments  in  the  testing  laboratory  In  all  respacts  with  all  of  tha  Impacts, 
vibrations  and  accelerations  ocouring  In  natural  sequences  or  combinations, 
then  are  we  deluding  ourselves  in  the  reliability  statements  we  make  and,  In 
effeot.  playing  a  numbers  game?  Or  can  the  statements  we  make  following 
tasting  of  bits  and  pieces  of  the  total  envlronmantai  picture  hove  a  real 
meaning  which  can  be  valuable  In  assessing  the  adequacy  of  a  given  design 
and  assuring  us  that  we  have  proceeded  far  enough  In  Its  development? 

Next,  If  the  reliability  numbers  we  obtain  can  have  some  meaning, 
li.  w  do  we  get  them?  What  is  the  best  reliability  testing  concept?  Do  wc, 
tor  example,  study  tlm  variability  of  the  balleiy  performance  IqvcIo  at  the 
specified  environmental  stress  levels  or  do  we  find  the  lallura  points  with 
iv.  "act  to  the  environments  and  then  determine  that  theue  failure  points  arc 
itutnclenUy  abo»e  the  roqriior..nnt  levels?  What  etotlntlual  mulhods  do  we 
use  or  develop  r  How  do  we  huddle  the  problem  of  the  types  of  distributions 
wo  are  dealing  with,  generally  unknown  particularly  when  dealing  with  high 
q  vlronmsntal  stresses?  What  are  the  consequences  of  saoura:.,,1  "art* in 
•  IlKL.'lbuttons?  How  do  we  express  the  reliability?  In  facing  these  questio.,.; 
w.:  must  buai  in  mind  the*,  we  don't  wunt  to  force  the  development  of  hnant 
winch  Is  far  beyond  a  reasonable  and  necessary  else  and  cont.  And,  In  our 
tenting  programs,  we  must  remember  that  sample  elites  must  bn  limited  by 
co"  of  the  Potteries,  seldom  less  than  one  hundred  dollars  a  piece. 

These,  then,  are  the  general  questions  about  reliability  which  wo 
urgently  need  answurs  to.  If  the  answer  is  that  ther»  Is  no  answer,  this 
,ou  is  important  to  know. 
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MONTE  CARLO  APPLICATION  FOR  DEVELOPING  A  DESIGN 
RELIABILITY  GOA.'.  COMPATIBLE  WITH  SMALL 
8AMPU.  REQUIREMENTS 

Ray  Hanthoook  and  Data  L.  Burrow* 

George  C.  Marshall  Space  Flight  Canter 
Propulsion  &  Vehicle  Engineering  Laboratory 
Teohnloal  &  Soientlfio  Stall 


SUMMARY.  Thla  report  desottbes  the  application  of  Monte  Oarlo  aim- 
ulation  to  the  construction  of  empirical  sampling  distribution  of  reliability 
estimate*  obtained  by  sampling  from  this  classical  stress. and  strength  (load 
and  failure)  distributions,  which  arc  assumed  Gaussian.  Typical  stress/strength 
distributions,  representing  specific  values  of  reliability,  were  stored,  in  a 
computer.  From  the  distributions  various  sample  sixes  were  taken  and  the 
resulting  estimate  of  reliability  computed.  Iteration  of  this  process  resulted 
in  the  construction  of  emplrloal  sampling  distributions  for  specific  values  of 
reliability  and  specific  sample  sixes.  Emphasis  was  placed  on  vary  high  values 
of  reliability  (.99  to  .99999999.)  and  on  very  small  aampla  alsaa  (2  to  B) 
beoausa  u  high  reliability  requirement  coupled  with  a  limited  number  of  teat 
articles  Is  commonly  imposed  on  epece  vehiole  development  programe. 

Since  the  eampling  distribution  of  reliability  estimate*  wes  found  to  be 
vwr /  sensitive  to  the  ratio  of  the  standard  deviations  ol  the  stress  and  strength 
distributions,  sampling  diatributiona  wara  constructed  for  specific  values  of 
thir  ratio.  Since  the  true  ratio  of  standard  deviations  will  seldom  be  known, 
tii..  variation  In  the  sampling  distribution  due  to  thle  ratio  Introduce!  e  eerily, 
weakness  into  the  appl loatirr,  of  the  analysis*.  This  weakness  La  dleoueeed 
fully  m  the  body  of  thle  repost.. 

It  la  concluded  from  this  Investigation  that  the  sampling  atsl.'.Hitlona, 
constructed  by  Monte  Carlo  simulation,  may  be  utlllxed  to  eld  the  designer 
in  establishing  e  design  reliability  goal,  place  a  confidence  ooufllolunt  on 
reliability  estimates,  and  to  determine  If  sample  stress  and  strength  dale 
don  ion  strata  s  a  specified  reliability  at  a  ipacified  confidence  level. 

The  primary  purpose  of  this  rsport  U  to  present  a  method  and  examples 
of  the  use  of  the  method.  The  ogives  given  In  this  report  may  be  used  In 
actual  application,  however,  not  Indescnminately,  since  the  ogives  contain 
small  Inaccuracies  duo  to  the  curve  fitting  procedure, 
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I.  INTRODUCTION. 

A.  Background.  In  large  specs  vehicle  development,  end  in  many 
otner  flaide  e»  wall,  a  designer  le  oiten  asked  to  design  his  equipment  knowing 
that  later  a  very  email  temple  of  Item*  Will  be  tested  to  determine  if  a  spaolflad 
minimum  reliability  la  demonstrated  at  soma  fairly  high  confidence  level.  The 
designer  is  often  quite  perplexed  at  to  the  methods  and  reliability  goals  ha 
should  utilise  In  formulating  his  design.  Hts  own  training  and  experience  make 
him  favor  the  safety  factor  approach,  and  yet  statisticians  and  reliability 
engineers  advocate  safety  margins,  atras s/strength  relationships  and  other 
statistical  approaches.  The  publications  of  Robert  Lutser  advocate  the  safety 
margin  approaoh,  (Ref .  1)  while  ARINC  Research  Corporation  under  oontract 
NASA  (Ref,  2)  has  advocated  the  uaa  of  itraai/strength  relationship*  aa  a 
reliability  prediotlon  technique.  There  are  many  other  papers  and  publications 
which  propose  the  use  of  statistical  variations  In  strata  and  strength  In  design 
and  subsequent  analysis  of  the  design.  The  application  of  a  single  distribution 
(strength)  and  a  reliability  bounds, y  (upper  limit  of  stress)  Is  discussed  In 
references  1  and  3 . 

&.  Scope,  This  Investigation  attempts  to  shed  tome  additional  light 
on  problems  concerning  the  use  of  stress/strength  statistics  In  design, 
reliability  demonstration,  and  confidence  limits.  It  Is  primarily  concerned  with 
the  solution  of  statistical  sampling  problems  which  have,  no  known  theoretical 
aviation  and  with  the  application  of  ths  data  provided  by  these  solution*.  In 
this  analysis,  vary  small  sample  slssi  In  the  rango  of  2  to  S  and  high  reliability 
value  irom  .99  to  .99999999  have  been  purposely  uaad  because  spaas  vehicle 
devourment  programs  are  restricted  to  small  samples  and  require. high  reliability 
valuer 


The  authors  wish  to  acknowledge  the  very  capable  asalntanoe  of 
Mesern.  Robert  Grafts,  Joe  Medlock  and  Matt  Blue  of  the  Computation  Laud¬ 
atory,  'vfSFC,  for  programming  the  Monte  Carlo  simulation  scheme,  and  of 
Mr.  B .  L.  Bombara,  Engine  Projeota  Office,  MSFC,  whoso  technical  advfcu 
and  suggestions  ware  very  helpful . 


II.  DESCRIPTION. 

A.  Analysis.  The  reliability  of  many  Heme  in  large  space  vehicles 
from  piece  parts  to  large  structural  elements,  can  be  eppropiletely  considered 
to  be  a  function  of  a  stress  distribution  and  a  strength  distribution.  A  stress 
distribution  la  defined  as  a  distribution  ot  stresses  to  which  the  population 
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of  items  will  bo  aubjoatod  In  actual  use,  end  a  strength  distribution  is  defined 
as  a  distribution  of  stresses  which  will  cause  failure  of  the  items.  This 
analysis  applies  to  the  general  problem  of  pitting  a  distribution  of  "what  an 
item  will  do"  against  a  distribution  of  "what  it;  is  required  to  do"  in  any  per* 
formanoe  parameter.  The  analysis  is  not  restricted  to  the  more  oommon  applica¬ 
tion  of  structural  stress/atrength  analysis. 

An  example  of  the  relationship  between  the  stress  and  strength  distri¬ 
butions  of  a  typical  situation  is  shown  in  Figure  1  (Figure*  era  at  the  end  of  this 
article).  The  distributions  ere  assumed  to  be  normal,  an  assumption  which  is 
retained  throughout  this  paper. 

A  randomly  chosen  value  of  itrasi  (X2)  subtracted  from  a  randomly 
chosen  value  of  strength  (X.)  gives  a  variate  from  the  strength  minus  stress 
distribution.  By  repeating  this  process  many  times,  e  distribution  of  strength 
minus  stress  may  be  iormed.  It  will  appear  aa  shown  in  Figure  2.  The  mean 
of  this  distribution  is  the  mean  of  the  strength  distribution  minus  the  mean  of 
the  stress  distribution.  The  standard  deviation  of  this  distribution  la  the  square 
root  of  the  sum  of  the  variances  of  the  strength  end  stresa  distributions.  At 
some  point  on  this  distribution,  stress  equals  strength  and  a  cero  point  appears 
on  the  Xi  -  X2  axis.  Since  any  negative  value  of  strength  mlnue  stresa  re¬ 
presents  a  failure,  the  area  below  the  zero  point  represents  unreliability,  and 
ih'j  urea  above  the  zero  point  represents  reliability.  Mathematically  this  may 
hi,  stated  as  follows  (Ref.  4); 


R  -  M*1  £  Xa) 


•IX,  - 

x2  -  MX1 

f  MXz 

-  M„i  +  Mx2) 

,(Xl  - 
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*  mX2 
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M*1  ) 
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In  aotufil  praotloe  tha  parameters  of  tha  atraaa  and  strength  distribution* 
will  rarely  be  known  and  must  ba  estimated  from  cample  dale,  Small  sample 
estimates  of  the  parameters  may  be  used  to  estimate  reliability  as  follows: 


table  of  areas, 


whtoh  can  bo  obtained  from  the  normal 


Slnoe  an  estimate  of  tha  reliability  of  a  particular  situation  can  now  be 
made,  the  next  logical  step  Is  to  describe  the  variations  expected  In  this  esti¬ 
mate  due  to  sampling.  In  order  to  make  use  of  the  stress/strength  statistical 
relationship,  the  sampling  distribution  of  reliability  estimates,  besed  on  this 
relationship,  must  be  developed.  Slnoe  there  was  no  known  theoratloal  solu¬ 
tion  to  the  description  of  this  variation,  a  computer  program  wee  developed 
using  Monte  Carlo  simulation  to  derive  tha  empirical  sampling  distribution  of 
tho  quantity 


Very  briefly,  this  simulation  technique  consists  of  ths  following  steps: 


(1)  8tore  hypothetical  ?'.Lcas  and  strength  distribution m  in  ihe  computer 

(2)  General*,  e  pseudo  random  number 

(3)  Use  this  random  number  to  get  o  random  value  of  strength  (Xj) 
repealing  this  procoso  N  times 

(4)  Compute  a  sample  mean  and  standard  deviation  for  strength 

(5)  Repeat  the  process  for  stress 


(6)  Compute 


(7)  Form  «  histogram  of  values  of  K,  which  represents  the  sampling 
distribution.  As  many  values  of  K  as  desired  may  be  obtained  from  the  pro¬ 
gram,  dependent  on  the  accuracy  desired.  1000  values  were  used  to  obtain  the 
Information  for  this  paper.  Th«  sampling  distribution  may  be  put  In  o  cumulative 
form,  termed  an  ogive,  In  order  to  be  able  to  read  K  values  corresponding  to 
selected  values  ol  p-.hahUtty . 
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B.  Results.  The  ftotors  which  Influence  the  sampling  distributions  \\ 
of  K  will  now  be  dlaousttd.  First,  semple  size,  of  coursp,  will  influence  it. 
An  empirical  sampling  distribution  must  be  generetid  for  ea*h  sample  s',ivu  that 
will  be  used  in  actual  practice,  To  give  an  idee  of  how  the  sampling  d',airl-‘:i! 
button  varies,  an  sample  siae  varies,  aome  ogives  for  various  sample  sires 
have  been  developed. 

The  ogives  in  Figure  3,  from  left  to  right,  represent  values  of  reita-1 
billty  from  .99  to  .99933999.  In  othar  words,  tha  first  ourva  to  tha  left  rar 
presents  the  variation  inharant  In  eatlmating  a  true  .99  reliability  using  a 
specified  sample  slza,  This  figure  represents  tha  aaae«*here,  Nx  .■  8  end 

Nx,  -8  l.a. ,  sample  slass  of  8  from  aaoh  of  tha  strength  end  stream  dlatrl- , 
buttons.  Ths  standard  deviations  of  stress  and  strength  am  equal  in  this  case, 
The  effeot  of  varying  these  atandard  deviations  will  be  discussed  later.  Ottser 
ogives  for  various  semple  sizes  end  equal  sigmas  appear  In  Figure  4  through  f. 

As  may  be  observed  from  Figure  3  through  8,  the  ogives  very  quite 
radically  with  semple  size,  especially  In. the  .vary  email  sample  sizes  used 
here,  In  order  to  use  this  type  empirical  data,  ogives  for  tha  specific  sample 
sizes  used  in  e  particular  application  must  be  developed. 

The  second  Important  factor  which  causes  the  sampling  distribution  to 
t.mry  la  the  ratio  of  the  atandard  deviations  of  stress  end  strength.  This  ratio 
is  dnflned  as  the  smeller  standard  deviation  divided  by  the  larger  stand'  rd 
deviation,  Early  In  thle  program  Monte  Carlo  results  Indicated  that  the  sigma 
,-a*ioihad  an  Important  offset  on  ths  variation  of  the  sampling  distribution,  and 
laior,  ths  theoretical  appilcnllnn  of  refarenco  5  served  to  verily  this  conclusion, 
Tha  results  or  the  theoretical  application  shewed  tnat  for  equal  sample  sizes 
the  degrees  of  freedom  of  tnt  non- central  t  distribution  is  a  function  of  sample 
!»i?,u  and  the  ratio  of  standard  deviation!  of  the  stress  and  stiength  ui-ir'biiHons. 
Figure  9  shows  the  variations  In  tha  sampling  distributions  for  various  ratios 
of  the  standard  deviations  with  a  fixed  sample  size  of  NXj  ■  i, ,  Hera 

only  the  upper  portion  of  the  distributions  are  .shown  most  applications 
vT.it  ba  concerned  with  high  confldsnos  and  because  tha  variation  Is  most  pro- 
ncuriood  In  this  area,  These  curves  wars  obtained  by  holding  all  vsrlaoles  In 
the  computer  program  constant  except  the  sigma  ratios,  Ths  sample  sizes  were 
equal  for  stress  and  strength,  As  the  figure  indicates  by  convergence  of  the 
curves,  there  Is  little  variation  in  the  aampling  distribution  of  "KM  attribu¬ 
table  to  sigma  ratios  below  the  60th  percentile  point  of  "K" .  Above  thle 
point  there  Is  considerable  variation;  the  higher  retlor  result  In  sampling 
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distributions  which. are  skewed  to  a  greeter  degree,  Although  ratloa  up  to 
1/100  ware  run  on  tha  Monte  Carlo  program,  the  praattoal  range  la  probably 
between  1  and  1/10  for  most  applications.  Even  at  tha  1/10  ratio,  however, 
there  la  a  large  variation)  therefore,  separate  curves  for  the  apeolfld  ratio  must 
ba  utilized  In  aatual  application.  A  weakness,  however,  In  this  procedure  la 
that  In  practice  the  actual  ratio  will  seldom  be  known  and  therefore  must  be 
estimated  from  sample  data,  An  F  test  for  the  ratio  of  two  varlanoes  oould  be 
used  to  establish  If  there  is  any  significant  difference  between  the  sigmas  and 
possible,  confidence  limits  for  the  ratio  of  two  varlanoes  oould  be  utilised  for 
estimating  tha  limits  of  tha  ratios,  Further  work  Is  desirable  in  developing  an 
approach  to  establishing  a  ratio  from  sample  data  whlah  oould  be  used  for  enter* 
ing  the  appropriate  set  of  curve*. 

Another  Importsnt  factor  to  oonalder  Is  tha  oeae  of  unequal  sigmas 
(stress  and  strength)  and  unequal  sample  sizes.  The  moit  skewed  oondltlon 
(long  tall  to  the  right)  In  the  sampling  distribution  of  "K"  will  rasult  when  the 
smaller  sample  Is  taken  from  the  distribution  which  has  the  larger  standard 
deviation,  For  Instance,  assume  that  tha  standard  devlstlon  of  strength  Is 
twice  that  of  (trees  and  a  sample  of  8  Is  token  from  the  stress  distribution 
and  5  from  the  strength  distribution,  I .e. ,  tha  smaller  sample  le  taken  from 
the  distribution  with  the  larger  slgme,  Whet  would  happen  If  a  reverse  proce¬ 
dure  ■  • used  end  the  large  sample  taken  from  the  distribution  having  the 
larger  sigma? 

lho  ogive  for  this  case  is  shown  In  Figure  10.  Tha  broken  line  represents 
the  egl'-  for  tha  large  sample  matched  with  the  large  sigma,  and  the  solid 
l Inn  repri.„ents  a  rerun  of  the  same  case  with  the  small  sample  with  the  large 
uigma.  As  can  be.  uc.cn,  ll.eie  is  a  significant  variation  In  the  two  conditions , 

It  is  concluded,  thcratore,  thst  this  <«  another  condition  which  must  bo  included 
in  the  Monte  Carlo  output  In  order  to  utilize  the  results  efficiently  and  aocuratsly. 
This  pt scents  no  problem,  however,  11  the  etgma  ratio  Is  known  elnoe  Monte 
Carlo  runs  for  the  required  conditions  can  be  made.  It  Is  being  presented 
merely  it  Illustrate  that  It  does  have  an  Influence. 

Thu  dtscuHston.  thus  far,  has  dealt  only  with  the  mechanics  of  obtain¬ 
ing  the  sampling  distribution  of  "K"  and  Its  variation  as  related  tu  epeoUlc 
factors  causing  it.  It  is  appropriate  to  discuss  the  application  of  such  date. 

C.  Application.  The  criteria  for  determining  whether  sample  data 
demonstrates  a  glvon  reliability  at  e  specified  confidence  will  be  developed 
In  terms  of  "K"  which  la  normal  deviate  corresponding  to  a  specified  area 
under  the  normal  curve. 
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Th*  basis  for  developing  the  demonstration  criteria  t »  ahnwn  t n  rigurw  u 
which  depict!  the  dietributiem  of  Xj  -  X2  end  Xj  -  X2  -  Kg  +  8*  *  . 

The  Z  shown  in  the  figure  La  •  normal  deviate,  the  area  above  whlohHobtalned 
from  normal  tables  of  areas)  represents  the  reliability  which  it  is  required  to 
demonstrate.  Kc  represents  a  normal  deviate  greator  than  Z  which  must  tie 
found  and  when  applied  will  assure  a  demonstrated  reliability  with  confidence  C. 

If  a  Kc  that  satisfies  the  following  inequality 

■  x2  -  K0VsXl2  *  a^  |  MXl  -  mX2  -  z  AXl2  +  <rX2a  )-  o 

(Rat.  3)  can  be  found  and  applied  as  a  orlteria,  a  decision  can  be  made  es  to 
whether  or  not  the  sample  data  demonstrates  a  specified  reliability  (Z).  This 
inequality  may  bo  reduced  to 

P,.  (Xx  -  X2  -  ®x2  i  «  »lnoe  -  M*2  -  i  ^  -  O 

in  this  nltuation,  because  the  Monts  Carlo  program  was  run  on  this  basis. 
Manipulation  of  this  insquallty,  as  follows,  mathematically,  gives  the  criteria: 

Pr(Xi  -  x2  s  k/8xi2  ♦  SX22  )  -  0 

°r  Y  -  X 

X1  *2 

M  y-  4  Kg)  ■  C 

V  >V  *  Kn 


Now  K0  can  be  found  by  referring  to  the  Monte  Carlo  developed 
ogives  for  the  conditions  of  the  problem  (speolfled  Z.  NXl<  NX2,  g-  ,  0’,ln), 

Once  Kc  is  found,  the  otiterle  for  demonstration  Is  as, follows: 


If  the  sample  quantity 


Ll 


x2 


A? ♦  **? 


£  V 


reliability  L  is  de¬ 


monstrated  with  th8  desired  confidence.  As  an  example,  suppose  samples 
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of  Nx  •  8,  NX2  -  8  gives  X:  -  80,000,  X2  -  60,000,  SXl  •  3000,  SX2  -  3000 

and  the  problem  was  to  determine  if  .9999  reliability  (Z  ■  3.71)  was  demonstrated 

at  the  90%  confidence  level.  The  90%  point  on  the  ogives  for  .9999,  N«,  ■  hT  ■  8, 

*1  *2 

O',.  -  O'  is  found  to  be  4.95  which  is  Kc  . 

X1  x2  c 

X.  -  X2 

- -  .4,7  therefore  since  4.7<4.95  reliability  .9999  is  not 


demonstrated  at  the  90%  oonftdenoe  level.  The  aoouraoy  of  the  answer  obtained 
le  dependent  on  how  dose  the  true  sigma  ratio  isito  unity,  since  it  was  assumed 
that  Of  “  O'  in  this  example. 

X1  x2 

Another  application  of  the  Monte  Carlo  results  is  the  establishment  of 
a  lower  confidence  limit  on  a  reliability  estimate.  The  appropriate  sot  of  ogives 
is  entered  with  the  reliability  estimate  (reliability  estimate  expressed  in  terms 
of  a  K  value)  and  a  confidence  is  read  for  each  reliability  value  represented  by 
curve,  If  sample  data  (NXj  -  NX2  *  8,  0*Xj  *  0^)  gave  a  reliability  astl> 

mate  expressed  in  terms  of  K*  3,5  for  a  given  sigma  ratio  and  a  given  sample 
sUe,  pionead  as  follows  to  arrive  at  a  lower  confidence  limit: 

(l)  Refer  to  the  series  of  ogives  (figure  12)  representing  the  sigma  ratio 
and  sample  sizes  applicable  to  the  problem. 

•.3)  On  the  horizontal  sea1*  locate  a  K  value  of  3 . 5  which  was  the 
sample  estimate  end  draw  a  vertical  line  through  the  point. 

(3)  Whero  the  vertical  line  intersects  a  curve,  read  a  confine  lieu  for 
the  reliability  represented  by  that  curve,  For  tha  example,  Figure  12  shows 
95%  confidence  In  ,99  reliability,  70%  confidence  in  ,999  reliability,  etc 
Ry  gnn.v atlng  a  sufficient  number  of  curves,  a  confidence  coefficient  for  the 
reliability  estimate  in  question,  or  at  least  close  enough  to  li  for  practical 
application,  can  be  obtained . 

If  in  the  initial  design  of  an  item,  a  designer  knew  exactly  ths  stress 
distribution  and  knew  exactly  what  strength  distribution  he  could  gat,  designing 
would  ou  a  simple  problem  and  there  would  bg  no  need  for  demonstration, 
However,  ihla  Ic  not  the  case,  and  the  designer  has  to  make  estimates  of  the 
dl  strlbutlons , 
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Assuming  thet  the  deslgnur  knew  exactly  the  strength  &  stress  distribu¬ 
tions  end  was  to  design,  knowing  that  a  small  sample  of  his  Itoms  was  to  be  11 
tested  later  for  demonstration  purposes,  it  would  be  to  hts  benefit  to  over- 
design  so  that  ha  would  have  a  good  ohanoe  say  90% ,  of  having  the  small 
sample  demonstrate  the  specified  reliability,  In  order  to  arrive  at  hbw  much 
he  should  overdesign,  ne  can  consult  the  Monte  Carlo  developed  ogives  and 
find  a  Kg  suoh  that  if  the  sample  K  Is  greater  than  K0,  the  sample  data  has 
demonstrated  the  required  reliability  at  the  desired  confidence,  Referenoe  to 
Figure  13  will  aid  the  reader  in  following  this  approach,  It  the  designer  wants 
a  90%  ohanoe  of  having  a  sample  demonstrate  the  required  reliability,  he  must 
design  to  a  reliability  represented  by  an  ogive  90%  of  whloh  if  above  the  Kq 
point.  This  logic  seams  to  ha  non  senslual  si  non  If  a  designer  knows  tha  dis¬ 
tributions  ha  aould  just  design  to  the  reliability  he  dailras  and  there  would  be 
no  point  in  a  demonstration  program)  however,  thlc  logic  oan  be  applied  to  the 
situation  where  the  deaigner  does  not  know  the  distribution*  but  has  some  know¬ 
ledge  of  them  from  experience  or  design  calculation.  Assuming  a  designer 
wants  to  design  so  that  he  has  a  90%  ohanoe  of  demonstrating  a  specific  re¬ 
liability  with  t>  specified  sample  size,  It  can  ba  concluded  from  th*  previous 
discussion  that  h*  must  design  to  e  reliability  above  that  which  gives  him  a 
90%  ohanoe  of  demonstrating  it  with  the  specified  sample  sla*.  How  much 
above  is  a  matter  of  engineering  judgement,  and  dapanda  upon  how  wall  the 
designer  thinks  he  oan  estimate  the  distributions .  This  conclusion  vividly 
point*  out  the  fact  that  e  design  goal  (inherent  design  reliability)  muat  ba 
hlghor  then  the  specific  reliability  which  le  to  be  demonstrated  at  a  high  de¬ 
gree  of  oonfidenoe,  There  aro,  of  ooursa,  other  faoteu,  whirh  Influence  the 
establishment  cf  reliability  goals.  Weight,  cost  and  performance  all  should 
Inf'.tenoe  design  decisions  and  must  be  properly  co.  iluored  an  tradeoff  factors 
against  the  statistically  duv.lopad  goal. 

Since  much  of  the  foregoing  discussion  of  reliability  has  Lssn  In  terms 
of  the  normal  deviate,  Z,  Figure  14  has  been  provided  to  enable  the  reader 
determine  Z  directly  from  e  numerical  value  of  unreliability  (l  -R) . 

III.  CONCLUSIONS  &  RECOMMENDATIONS. 

A,  Conclusions,  This  Investigation  has  rovealed  that  the  statistical 
information  afforded  by  very  small  samples  from  a  stress/strength  situation 
(even  as  low  as  2)  can  ba  useful  to  the  designer  and  the  reliability  engineer. 

It  has  also  revealed  that  the  sampling  distribution  of  reliability  estimates 
made  by  taking  sample  data  from  tha  stresB/strength  distribution*  is  very  sen- 
olttvo  to  sample  size  nnd  the  ratio  vi  ii.m  standard  deviations  of  the  str  si?  and 
etiength  distributions. 
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It  Is  concluded  that  the  empirical  sampling  distribution  of  reliability 
estimates  can  ba  utilized  by  the  designer  or  the  reliability  engineer  as  follows: 

1.  Reliability  Demonstration-Given  sample  data  from  stress  and 
strength  distributions,  a  determination  can  bo  made  as  to  whether  the  sample 
data  demonstrates  a  specified  reliability  at  a  chosen  oonfldenca  level, 

2.  Establishing  Oonfldence  Coefflclents-GLven  sample  data  from  a 
strsas/strength  situation,  confidence  coefficients  for  various  values  of  relia¬ 
bility  can  be  estimated,  limited  only  by  the  number  of  curves  that  have  been 
generated  by  Monte  Oarlo , 

3. '  Recommend  Design  Goal-One  can  establish  and  recommend  to  the 
designer  a  design  goal  such  that  If  he  designs  to  It,  a  pre-ohossn  sample  will 
demonstrate  a  specified  reliability  at  a  chosen  confidence  level,  a  specified 
percent  of  the  time  , 

B.  Recommendations  for  Future  Investigation.  In  view  of  the  number 
of  promising  applications  discovered  during  this  Investigation  and  the  weak¬ 
nesses  and  limitations  that  are  Inherent  In  the  nature  and  extant  of  this  ana¬ 
lysis,  It  Is  believed  that  the  following  areas  are  worthy  of  further  Investigation: 

1 ,  Since  the  ratio  of  standard  deviations  of  the  stress  and  strength 
dlstrlnutlona  has  a  large  affect  on  the  variance  of  the  reliability  sampling 
distribution  (Distribution  of  "K")  and  slncu  In  practice  this  letlo  will  seldom 
be  kiiov  -  but  must  be  ostlmated  from  sample  data,  a  method  eliould  be  derived 
for  ea.aullshlng  a  ratio  which  could  he  used  for  entering  the  app"  prlate  set  of 
empirical  curves. 

2,  Extension  of  the  analysis  to  various  combinations  of  dlfferm,:  ♦yean 
of  distributions,  other  than  normal,  should  be  Investigated  by  the  Monte  Carle 
process  nlnoe  m  these  areas  not  even  approximation  to  theoiattoal  solutions 
are  available , 
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EXACT  CONTINGENCY  TABLE  CALCULATIONS 


Dorothy  Burg,  Murley  Leyton’1',  and  Clifford  J.  Maloney 
Division  of  Biologies  Standard* 

National  Institutes  of  Health 


1.  INTRODUCTION.  The  great  Dane  is  known  throughout  the  elviMsed 
world  lor  his  famous  rhetorical  question;  "To  B  or  not  to  B  But  he 
had  a  predecessor,  equally  famous,  who  2000  years  ago  and  more  divided 
the  universe  not  surprisingly  Into  the  even  prior  dichotomy  A  and  not-A, 
These  pioneering  efforts  were  continued  in  many  lands  while  Europe  slept 
with  the  result  that  not  203  years  ago  science  was  able  to  divide  the  universe 
not  only  into  A  and  not-A,  but  also  simultaneously  into  B  and  not-B,  We 
show  this  model  in  table  1,  except  that  we  have  eubetituted  the  letter*  R 
and  C  (from  row  and  column)  for  the  Arietotelian  label*  A  and  B,  Every 
element  of  the  universe  of  discourse  is  presumed  to  be  unambiguously 
assigned  to  one  of  the  four  cells  in  the  body  of  the  table, 

We  need  not  tell  this  audience  that  the  probability  of  any  given  set 
of  four  cell  frequencies  among  thn  »-,t  of  all  four-fold  table*  with  speci¬ 
fied  row  and  column  total*  is  given  by  the  hypergeometric  distribution 


U) 


c  !  C  t  r  !  R  ! 
f!  g!  h'.klN! 


Accordingly,  reference  (19)  can  be  used  to  obtain  significant*- 
level*  for  vesting  observed  insults.  Table*  for  exact  probabilities  where 


N  -■  R  +  r  and  v  R  *  30 


ind  less  completely  where 


31  -  R  -  40,  r  <■  R 


*'  Now  at  tiie  School  of  Public  Hygiene,  Johns  Hopkino  University 
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have  recently  appeared,  (6).  More  exact  and  more  complete  table*  are 
available  In  the  UMT  file  of  Mathematic*  of  Computation  at  the  David 
Taylor  Model  Basin,  Washington  23,  D,  C,  ,  covering  With  (6)  the  range 

r  *  R  &  43 

with  table  entrie*  to  four  place  accuracy,  Table*  for  2X3  contingency 
table*  in  the  ipecial  cue  that  all  two  cell  marginal  total*  are  equal,  in 
the  range  3  (1)  20  at  level*  of  significance  0.05,  0.025,  0.01,  0.001  have 
recently  appeared  (3),  Robert  ion  (23)  ha:  published  a  computer  program 
by  the  use  of  which  exact  probabilities  can  be  calculated  tor  any  2X2 
contingency  table.  The  computer  program  for  handling  the  2X3  case 
presumably  i*  available  from  the  author*  (3). 

Freeman  and  Halton  (7)  have  published  a  computing  method  for 
calculating  exact  contingency  probabilities  for  tables  of  any  else.  They 
treat  also  the  calculation  of  interaction  probabilities.  Unfortunately  the 
procedure  is  more  theoretically  than  practically  adequate, 

It  was  early  realised  by  Karl  Pearson  (23)  that  this  probability  could 
be  approximated  by  the  chi-square  distribution  provided  the  individual 
call  entrie*  were  not  too  smell,  Yates  (32)  supplied  a  correction  which 
would  make  uee  of  the  chi- square  approximations  appropriate  for  smaller 
cell  entries  than  would  otherwise  b*  possible,  but  pointed  out  that,  wlvre 
thr  cell  entries  were  extremely  unequal,  evsn  this  correction  would  not 
be  adequate  until  the  minimum  cell  entry  of  th*  table  should  become  qui»‘ 
large, 

A  long  series  of  authors,  among  whom  Haldane  (9)  was  particularly 
prominent,  have  studied  the  question  of  developing  adequate  eppro.i'ma. 
tirns  to  exact  contingency  probabilities  when  the  chi-aquare  is  not  useful. 
Others  have  considered  various  other  aspects  including  studies  oL  the 
power  function  of  the  lest,  choice  of  tests  for  interactions,  an<*  measure* 
of  the  strength  of  the  association  between  the  factor*  when  independence 
is  not  present,  Particularly  important  from  th*  standpoint  of  the  -xuct 
calculation  of  contingency  table  probabilities  is  th*  paper  of  Kimball  (16) 
where  it  is  shown  that  the  partition  of  contingency  table  chi-equare  can 
be  reduced  to  the  repeated  calculation  of  2  X  2  tables,  A  selected  list 
of  these  papere  Is  provided  in  the  bibliography. 
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2.  FIRST  APPROACH,  A  table  which  arose  in  our  work  and  which 
actually  gave  riio  to  this  study  Is  given  in  table  2.  A  set  of  data  in 
which  the  reality  of  the  three  factor  interaction  was  the  point  of  Interest 
is  shown  in  table  3.  The  entries  in  table  2  are  (some  of  them)  too  small 
to  apply  the  simple  chi-square  test  blindlyt  yet  others  are  much  too  large 
for  manual  application  of  the  Freeman-Halton  procedure!  The  computer 
program  for  the  2  X  3  case  (3)  had  not  been  announced  (terms  of  avail¬ 
ability  were  not  stated)  at  that  time,  and  a  computer  program  for  the 
2X2X2  case  has  not  yet  appeared, 

In  an  effort  to  deviso  a  procedure  for  exact  contingency  table  cal” 
culations  which  might  be  generalised  to  the  general  R  X  C  ease  and  which 
would  be  adapted  to  computer  calculation,  formula  (1)  for  the  probability 
of  the  particular  set  of  cell  entries  which  appear  in  table  (1)  was  rear* 
ranged  in  the  form 


which  brings  out  somewhat  mors  clearly  that  the  formula  is  a  composite 
of  binomial  coefficients,  The  generating  function  can  hence  be  shown 
in  figure  (1).  [Figure  1  follows  table  3  near  the  end  of  this  article  J , 

It  is  well  known  that  the  exponent  on  X  plus  the  exponent  on  Y  is  a  4 
constant  for  all  terms  of  a  binomial  expansion,  In  figure  (1) 

f  +  g  ■  r  and  h  +  k  ■  H. 

Further,  if  re  multiply  th»*  ‘wo  binomials  term  b>  term  then  for  certain 
terms 


t  +  h  ■  c  and  g  +  k  ■  C 

nod  the  coefficient  of  this  term  is  prscistly  ths  numerator  of  the 
probability  of  the  table  in  queetion,  For  th»  particular  cate  of  a 
product  of  two  binomial*  th*  product  nf  one  term  from  the  firet,  !>',  a 
unique  term  from  the  second  yields  the  only  pair  whose  exponents  meet 
the  marginal  conditions,  and  hence  is,  by  itself,  (proportional  to)  the 
probability  of  that  particular  set  of  cell  frequencies. 
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This  luggeite  •  very  simple  computing  rule  for  calculating  exact 
probabilities  in  a  2  X  2  contingency  table.  One  write*  out  the  numerator* 
of  the  binomial  expansion  of  the  smallest  marginal  total  of  the  table  whose 
probability  i»  desired.  Lot  this  smallest  total  be  designated  by  r.  Then 
the  r  +  l  terms  of  ths  expansion  of  this  binomial  are  written  in  a  row.  In 
our  notation  R  will  then  j*  greater  than  c,  Wejiaxt  writs  out  in  a  row 
beneath  the  preceding  terms  ths  r  +  1  terms  of  jM  in  order,  starting  with 
«  «  r  as  the  value  for  h  under  ths  right  most  element  of  the  preceding 
expansion  and  proceedings  to  the  left.  These  two  rows  are  next  multi* 
plied  element  by  corresponding  element  to  give  ths  r  e  1  numerators  of 
the  complete  probability  distribution  of  the  given  2X2  table.  These 
terms  are  summed  and  saoh  divided  by  that  sum  to  yield  the  numerical 
value  of  the  probabilities.  The  sum  of  the  probabilities  tor  thoaa  terms 
equal  or  more  extreme  than  the  obeerved  table  give  the  desired  exact 
probability  by  which  the  independence  of  its  main  factors  is  tested, 

The  procedure  may  be  illustrated  by  an  example.  Let  r  ■  5, 

R  •*  10,  c  ■  7,  C  *  8.  We  set  out  ths  complete  expansion  fojr.the  hi*, 
nomial  of  the  fifth  degree  and  under  it  ths  six  terms  from  [  7}  to  jj 
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For  a  two-tailed  test,  ths  probabilities  for  testing  any  observed 
table  where  ths  marginal  totals  are  3,  10,  7,  and  8  is  than  a  function 
of  its  entry  in  ths  call  defined  by  ths  3  and  7  marginal  totals  only. 
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P  ( 0  E2T« 


45 

6435 


163 

THs 


763 

6435 


1815 

TZVT 


3915 

TiTiT 


64  35 


A  one-tailed  te»t  for  a  2  X  2  table  would  be  exactly  analogous.  Exact 
probability  calculation)  are,  of  courae,  not  needed  if  the  entriei  In 
every  cell  are  large.  If  one  marginal  total  ia  email  and  one  large 
(on  the  aame  margin,  aay  r<  <  R)  the  calculation!  can  be  greatly 
almpllfied  by  expreaaing  each  in  terma  of  one  of  their  number.  For 
example,  if 


ic  -  r 


then 


(3) 


e  ■  r  +  1 

C  -  1 
c  -  r  +  2 

C  -  1 
c  -  r  +  2 


R 

r  +  1 


C  -  r  +  1 


As  an  exarr  le  of  theae  calculation),  consider  table  (4).  Here 
r  =>  9,  R  »  681,  0  *i  *  <).  Since  Xj  will  eventually  cancel  out , 
aet  it  equal  to  one.  The  calculation)  follow: 
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9 

36 

84 
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84 

36 

342  v 

34  3 

344  v 
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346  „ 
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348  v 
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340  *8 

339  i 

338'  X6 

337  *5 

-33rX4 

'35  *3 

334  *2 

333 

In  systematically  calculating  the  probabilities,  a  triple  product  1*  required) 
each  element  i«  multiplied  by  it*  right  hand  neighbor  a*  well  a*  by  the  number 
ju*t  above  it  in  the  preceding  row,  The  term  and  cumulated  frequencies  a* 
function*  of  f,  the  entry  with  imalleit  joint  marginal  total*  can  be  expressed. 


f 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

p(0 

0.  002 

0,019 

0.075 

0. 171 

0.251 

0.  244 

0,157 

O 

o 

o* 

0,018 

0.002 

Hi) 

0.  002 

0.021 

0.096 

0,  267 

0,  518 

0.762 

0.918 

0,983 

0.  99B 

1 

The  procedure  can  be  systematically  extended  to  2  X  n  tables,  though  the 
work  involved  quickly  increase*  with  n.  The  general  formula  for  the 
probability  of  a  specified  2  X  n  contingency  table  with  fixed  marginal 
totals,  a*  given  for  example  by  Freeman  and  Helton,  can  be  rearranged,  as 


(4) 


Where  N  1*  the  table  total)  r,  *,  t,  ,  ,  .  Rare  the  n  marginal  total: 

in  th;  long  direction;  and  C  is  one  of  the  2  marginal  total*  in  the  other 
direct-ion.  In  thin  form,  it  is  obviou*  that  the  calculation*  are  of  the 
lanin  type  as  those  required  in  the  2X2  cnee  but  that  now  the  product 
of  an  appropriate  lerm  from  each  of  n  binomial  expansion*  will  be 
required.  An  exteneion  of  the  generating  function  of  (3)  make*  clear 
the  terms  from  each  binomial  expansion  Mmissabls  in  each  product. 
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For  the  2X3  caue,  the  eelcuUtions  proceed  ei  follow*.  Aatume 
again  that  _f  i*  the  *malle*t  entry  in  the  table  and  that  r  and  c  are 
the  *malle*t  marginal  total*  for  row*  and  column*,  (Of  oourae,  thi* 

I*  no  reetriction  on  the  applicability  of  the  method).  Now  f  will  range 
from  0  to  r  (aasumed  lei*  than  c).  Each  value  of  f  will  detormin* 
one  call  of  the  2X3  table  exactly  and  thu*  reduce  the  remaining  cell* 
to  the  form  of  a  2  X  2  table  with  adjuated  marginal  totale  (on  the  3 
cell  margin), 

Consider  a  2  X  3  table*  in  which  the  3  cell  margtne  are  3  and  5 
and  the  2  cell  margin*  are  2,  3,  and  9.  Then  f  can  be  0,  1,  and  2. 

W*  mud  calculate  a  2  X  2  probability  dlitribution  for  each  of  the*e 
value*.  Let  now  b  play  the  role  of  f  in  tho  reduced  table.  It  may 
take  the  value*  0,  1,  2,  and  3  (aince  the  amalleet  remaining  total  1* 

3).  Hence,  in  exact  parallelivm  with  our  pravioue  calculation*,  when 
f  «  0,  the  qualifying  marginal  total*  are  3  and  3,  to  that  for  f  •  0,  a* 
b  range*  from  0  to  3,  h  take*  the  value*  3,4,  3,  and  2.  In  tabular  form 


b  0  1  2  3 

f(b)  13  3  1 

f(h)  1  9  10  >0 

l  19  30  To" 


Iheia  are  the  numerator*  of  the  probabilitle*  a*  they  aland,  einca 
tl-o  factor  from  the  binomial  expansion  ofj  1*  unity.  When,  next, 

f  take*  the  v»'n*  1,  the  marginal  total  5  la  reduced  to  4  eo  that 
the  calculation  become* 

b  0  1  2  3 

f(b)  1  3  3  J 

f(h-l)  3  10  10  9 

S  "W  30~  T“ 


* 


Table  5, 
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Now,  however,  the  numerator  In  the  expansion  of  j  J)  is  2,  so  that 
each  of  the  above  entries  must  be  doubled  to  yield 

10  60  60  10 


Finally,  when  f  =*  2,  the  only  remaining  value,  the  contribution  from 

ie  again  unity  and  we  have 

*  h  0  1  2  ? 

«{b)  13  3  1 

f(h  ■  2)  10  10  5  1 

10  30  IT  1 


These  final  products  are  proportional  to  the  complete  set  of  probabil¬ 
ities  for  admissable  tables,  They  may  be  set  out  as  in  table  (6).  When 
divided  by  their  total  these  entries  are,  of  course,  the  probabilities 
thsmielvss,  Extension  to  2  X  4  and  higher  tables,  while  increasingly 
laborious,  is  straightforward,  Another  option  exists,  however.  From 
equation  (4)  it  is  obvious  that  ths  equation,  being  a  running  product  of 
binomial  coefficients  can  bs  broken  at  an£  point  in  the  reduction  proceee 
rut  merely  after  ths  first  column,  as  was  done  above.  Thus,  a  2  X  4 
could  be  "factored"  into  two  2X2  tables  rather  thun  successively  into 
two  1  X  2  and  one  2X2  taSie,  A  computer  routine  should  he  easy  to 
devise  to  handle  all  cases  up  to  3  X  6,  Further  work  along  this  line 
h».s  been  temporarily  held  up,  however,  since  the  approach  seums 
lefh  promising  for  tables,  both  dimension*  of  which  exceed  two. 

3,  GENERAL.  A  X  C  TABLE  The  approach  ft dopted  in  Section  II, 
which  treated  several  Instances  of  the  2  X  n  contingency  table  e?»nt 
probability  calculations  was  to  exploit  the  fact  that  the  calculation* 
could  be  reduced  to  the  product  of  binomial  coefficient*,  The  general 
R  X  C  tabic  may  he  discussed  in  term*  of  a  3  X  3  table  to  simplify 
the  notation,  An  assignment  of  symbols  is  shown  in  Tshie  (7),  The 
probability  for  any  specific  set  of  numerical  values  of  the  symbol*  is 
well  known  to  bsi 
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In  a  manner  exactly  analogous  to  the  reasoning  used  in  the  2X2 
case,  this  can  be  reduced  to 


(6) 


K 


Rj,  Rj 

d!  ell!  “  *  iThTU 


so  that  the  generating  function  becomes 

3  R. 

(?)  a  •  n  (x  +  y  +  a)  1 


This  approach  could  serve  as  a  straightforward  method  of  calculating 
all  probabilities  for  any  3X3  table,  and  could  be  extended  in  an 
obvious  manner  to  the  general  R  X  C  case, 

The  method  suffers  from  two  drawbacks,  The  number  of  possible 
terms  grows  rapidly,  so  that  the  amount  of  calculations  grows  even 
.'aster,  Most  of  these  tables  will  have  extremely  small  probabilities, 
due  to  their  very  number,  Hence,  it  seoms  wise  to  devise  a  method 
of  approach  in  which  only  tables  of  appreciable  probability  will  appear. 
This  will  be  easy  to  insure  provided  (a)  the  various  tahlee  ran  be 
calculated  in  the  order  of  decreasing  probability  and  (b)  a  running  s,.-' 
c-f  the  probabilities  of  all  tables  calculated  so  far  le  maintained.  By 
netting  a  criterion,  say  0.99999,  and  terminating  calculation  when 
this  total  is  rsached,  it  is  obvious  that  the  Iasi  criterion  is  easy  to 
ensure,  Furthermore,  even  If  a  few  tables  ot  low  probability  are 
calculated,  no  harm  is  done  other  than  the  loss  of  machine  tints  s.i 
that  the  method  of  ordering  the  calculations  need  not  be  perfect.  A 
drawback,  if  perfect  ordering  is  not  schisved,  is  that  additional 
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machine  time  muet  be  devoted  to  sorting  to  make  the  final  output 
useful. 

The  procedure  adopted  in  our  current  approach  in  composed  of 
two  portions,  first,  a  single  table,  known  to  be  of  high  probability 
is  constructed,  further  tables  are  constructed  as  perturbations  in 
the  entries  of  this  table  so  selected  that  the  resulting  table  is  also 
likely  to  be  of  high  probability.  The  starting  information  is  the  set 
of  row  and  column  totals,  Hence  the  exact  probabilities  for  an 
R  X  C  contingency  table  is  a  table  of  R  +  C  -  1  entry.  Non-computing 
implications  are  discussed  below.  It  is  from  these  R  +  C  given  values 
that  we  construct  three  sets  of  tables  needed  in  the  calculations  and  in 
the  output,  first,  as  noted  above,  the  initial  complete  table  of  known 
high  probability  is  constructed  as  the  only  member  of  the  first  set.  It 
must,  howsvsr,  always  bs  held  in  memory  whereas  the  membere  of  the 
following  sets  may  be  sent  to  tape  when  calculations  on  them  are  com¬ 
pleted.  Tha  calculation  uf  this  basic  high  probability  table  is  done  by 
first  calculating  the  "expected"  cell  entries  in  tha  usual  manner  in 
applying  the  chi -squats  method: 


(?) 


These  entries  are  in  general  not  integers.  The  nsxi  step  is  to  reduce 
all  of  them  to  integral  form,  but  in  tuah  a  manner  that  they  sum  to  thn 
given  marginal  totals,  Thn  second  class  of  R  X  C  tables  consist  of 
tables  of  adjustment*  by  ir  »v.\*  of  which  other  contingency  tables  with 
the  speclfiud  row  and  column  totals  arc  formud  by  addition  of  the 
adjustment  matrix  to  the  original  starting  or  basic  matrix.  T**  this 
purpose  the  raws  and  columns  of  the  adjustment  matrix  itaalf  muet 
cum  to  aero,  Individual  entries  may  be  email  positive  or  ns^ative 
Integers  or  zero.  Thn  third  clsss  of  matrices  is  composed  of  these 
sum  matrices  themselves,  l.c.  the  set  of  sdmissaUs  matrices  other 
then  the  starting  matrix  itself.  This  is  ths  sat  of  matrices  that  mvst 
appear  in  the  output  if  it  is  to  bs  in  most  convenient  form. 

This  approach  has  just  now  (July  1964)  bean  programmed  for 
the  general  3X3  caet  so  that,  together  with  ths  approach  of  tha 
preceding  section,  we  should  now  bs  in  s  position  to  give  exact 
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contingency  table  probabilities  for  any  2  X  n  or  3  X  3  contingency 
table,  An  actual  3X3  table  required  10  second*  of  Honeywell  BOO 
computer  time  eo  that,  in  this  range  *t  leaet,  the  approach  looks 
hopeful,  In  th*  current  program,  even  for  the  3  X  3  case,  we 
compute  ail  admissable  tables  and  have  so  far  made  no  attempt 
to  minimise  memory  requirements  or  computer  time,  As  the  pro¬ 
gram  is  extended  to  higher  values  of  R  and  C  thin  will  become  more 
necessary,  On  the  othsr  hand,  as  the  dimensions  of  ths  table  increase, 
th*  possibilities  of  adsquate  approximating  formulae  increase.  The 
list  of  reference*  at  ths  end  of  this  paper  will  indicate  that  no  possi¬ 
bility  of  modifying  the  simple  chi-square  formula  to  extend  the  lower 
range  of  tabular  cell  frequencies  has  been  overlooked.  Unfortunately, 
no  means  exist*  for  choosing  between  them  in  the  absence  of  exact 
probabilitiee,  One  purpose,  which  we  hope  our  etudloi  will  serve, 
le  to  permit  euch  a  definitive  evaluation  of  the  available  approximating 
formulae, 

The  nub  of  the  present  computer  technique  le  the  fornhatlon  of  the 
eet  of  adjustment  matrices  in  the  order  of  decreasing  probability. 

This  last  is  rsquired  if  th*  calculations  art  to  bs  truncated  when  all 
table*  having  appreciable  probability  have  been  calculated,  Now  the 
rows  and  columns  of  every  adjustment  matrix  muet  sum1  to  aero. 

This  means  that  ths  sum  of  ths  positive  adjustments  muit  be  balanced 
by  ths  sum  of  ths  negative  adjustments,  This,  in  turn,  means  that 
the  minimum  number  of  adjustment*  in  any  row  or  column  (if  It  contains 
an  adjustment  at  all)  muit  be  two.  On  the  other  hand,  any  tabU  formed 
by  the  minimal  number  oi  minimal  adjustments  (±  1)  would  seem  to  be 
itself  a  high  probability  c,it.le,  To  exploit  these  properties  systematic¬ 
ally  it  is  intended  to  so  ...rang*  thn  row*  and  column*  of  the  table  that 


(9) 


Ri  4  Rl  +  1 


<  C 


t  +  1 


for  all  i  and  j,  which  is  obviously  no  restriction'  on  ths  gsnsrs'lty 
of  th*  method.  This  can  always  h*  done  as  the  row  and  column  sums 
are  the  input  to  th*  calculation,  Ssoond,  all  possible  corrections  of 
±  1  in  at  most  two  cells  of  any  one  row  and  column  will  be  calculated 
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first.  Then  all  in  at  moat  four,  then  six  and  to  on  untli  every  cell  ii 
adjunted  up  or  down  by  one  unit.  If  the  probability*  of  these  table*  do 
not  sum  to  the  cut  off  criterion,  then  all  oases  in  which  one  cell  la 
adjusted  by  two  unit*  will  be  similarly  calculated  ayitematlcally, 
During  any  on*  of  these  minor  cycle*  it  may  prove  worthwhile  to  intro¬ 
duce  an  individual  table  probability  test  equal  say  to  10*”  so  that  when 
a  table  with  this  low  a  probability  is  encountered  the  minor  cycle 
(shift*  in  the  row  and  or  column  of  the  adjustment)  is  terminated  and 
the  next  adjuitment  formula  applied,  In  the  caee  of  the  2X2  table, 
it  is  not  difficult  to  distinguish  between  a  one-tailed  end  a  two-tailed 
test  --  despite  the  fact  that  thi*  1*  seldom  dona,  In  the  general  R  X  C 
table  it  becomes  necessary  to  select  among  the  possible  definition! 
of  a  one-tailed  teat,  However,  whatever  the  definition,  from  a  com¬ 
puter  standpoint  it  reduces  to  deciding  what  terms  to  Include  In  a 
summation  of  probabilities.  Tor  this  reason  ws  hop*  to  print  out 
probabilities  for  individual  tables  and  rtudy  the  ssnsitivity  of  changing 
the  definition  on  acceptance  levels,  The  results  of  this  work  will  be 
reported  subsequently, 
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Symbol  luiitnmcne  tor  2  x  2  contingancy  Mbit, 


Tibia  2 


Population 

X 

II 

III 

Traatad 

23,973 

101,637 

56,613 

Infactad 

3 

49 

23 

Typical  actual  cast  of  vaoclna  trial  raaulta  In  thrae  traatad  citlaa 


Actual  2x2x2  contingency  table  data 


Figure  1.  Generating  function  for  Individual  table  frequenelae  of 
2x2  contingency  table. 


Tab  la  4 


Hypothetical  2x3  tabla.  For  explanation,  »«e  taxt 
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DESIGNS  FOR  THE  SEQUENTIAL  APPUCATIGN  OF  FACTORS1 
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Soma  expert  mental  plane  that  ara  appropriate  whan  the  level*  of  the 
factor*  of  the  2n  factorial  experiment*  are  applied  eaquentlslly  aro  presented. 
Theae  plan*  permit  the  eattmatlcn  of  the  main  effect*  and  two-feotor  Interact* 
Ion*  of  th«  2n  experiment  and  alao  the  effect*  of  the  different  possible  order* 
Inga  of  the  faotora. 


INTRODUCTION  ,  When  an  experimental  procedure  Involve*  the  deter¬ 
mination  of  the  offset*  and  interaction*  of  several  factors,  a  considerable 
advantage  Is  gained  If  the  experiment  1*  designed  so  that  the  effect  of  changing 
eny  on*  factor  can  be  assessed  Independently  of  the  other  feotor* ,  The  effect* 
of  all  factors  on  a  characteristic  of  Interest  may  be  investigated  simultaneously 
by  varying  each  factor  so  that  all,  or  a  suitable  subset  of  ell,  possible  com* 
blnatlons  of  the  factors  are  considered,  An  experiment  In  which  this  proce¬ 
dure  Is  used  Is  known  as  a  factorial  experiment.  A  factorial  experiment  con¬ 
sists  of  applying  the  ohosen  combinations  of  factors  to  the  experimental  units 
!n  a  random  manner  and  recording  the  yield  of  the  response  varl&ble. 


The  standard  statistical  analyses  of  factorial  experiments  are  appro¬ 
priate  when  the  factor  Levele  of  any  treatment  combination  are  simultaneously  | 
applied  to  an  experimental  unit .  Thera  ara  many  altuatlona  whan  It  Is  either  I 

impossible  or  impractical  to  apply  the  faotors  simultaneously  to  each  experl-  I 

msntal  unit,  fn  *.!?*•  clL-atlons  the  feotor  levels  may  b<*  applied  eequentlelly, 

If  ihei*  Is  but  possible  wav  to  order  the  sequence  of  factors,  or  If  the 
ordar  in  which  the  factors  are  applied  does  not  affect  the  reaponar*.  the  experi¬ 
ment  oan  be  analysed  as  If  the  factor*  were  applied  simultaneously .  Howuv  *r, 
it  ihe  faotors  can  be  applied  In  several  possible  orders,  and  If  the  response 
due  to  tha  applioatlon  of  a  combination  of  faotors  depends  upon  the  particular 
ordar  In  which  tha  factors  are  applied,  th*  experiment  should  bs  planned  so  as 
to  permit  an  evaluation  of  tha  order  effects .  Th*  order  In  which  factor*  a;«  ap¬ 
plied  to  experimental  units  msy  sffeot  the  response  due  to  differences  in  residual 


1 T h l ■  resoaroh  was  supported  by  a  grant  from  the  U,  S.  Army  Research  Offloe- 
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or  carry-over  affect*,  A  residual  affaot  will  occur  whan  a  factor  t*  appllad 
to  an  axparlmantal  unit  befora  tha  affaot  of  a  previously  applied  factor  ha* 
worn  off.  Thu*,  it  ia  quite  po* tibia  that  tha  application  of  factor  A  bafora 
factor  3  will  laad  to  a  differant  ratponaa  than  the  application  of  factor  A 
after  factor  B. 

Thl*  paper  1*  concerned  with  the  construction  and  analysis  of  axperl- 
menta.l  designs  for  the  sequential  application  of  the  factors  of  a  2n  factorial 
arrangement.  Throughout  the  paper  It  will  be  assumed  that  residual  affects 
are  permanent ,  l.a. ,  if  the  effects  of  two  faotors  are  order  dependant,  tha 
application  of  one  factor  will  influence  the  affect  of  tha  saoond  factor,  no 
matter  how  many  other  factors  are  appllad  between  them. 

Tha  experimental  plans  davalopsd  in  this  paper  may  be  quite  useful 
whan  tha  faotors  era  environments.  In  laboratory  simulation  experltninta  it 
is  often  Impossible  or  uneconomical  to  simultaneously  apply  all  tha  environ¬ 
ments  to  which  an  itam  may  bu  subjected' in  actual  use.  It  the  factors  are 
applied  sequentially  In  a  simulation  experiment,  Important  information  on  the 
residual  affects  can  b*  gained  by  varying  tha  order  of  the  sequence.  If  there 
are  alternative  sequences  of  factors  to  which  an  item  is  subjected  in  actual 
use,  one  may  determine  the  order  of  th*  sequence  which  is  least  detrimental 
tc  i ho  item  under  test,  by  varying  th*  order  of  the  sequence  in  simulation  ax- 
perlmants.  These  plans  may  also  prove  useful  in  experiments  where  items 
aru  treated  with  a  sequence  of  ohemioais  to  improve  some  chereoterlstic 
(herdness,  rust  rsilstanoe,  etc.)  of  th*  Items. 

PLANS  WITH  TWO  fACTOUfl  DEPENDENT  OK  ORDER.  In  thts  soUlon 
wa  will  oonsiUui  experimental  p!:r,j  for  the  2"  factorial  arrangements  where 
the  order  ot  only  two  of  the  n  factors  will  affaot  the  response.  Some  of  the 
pis  nr  are  constructed  in  suoh  a  mannar  that  all  factor  affects  of  interest  arid 
the  order  effect  ere  orthogonally  estimable.  For  some  plans  tha  factor  and 
order  uffeou  and  tha  Interactions  are  partially  correlated  It  Is  assumed  that 
ihu  luw  level  of  a  factor  (0)  indicates  that  the  factor  we*  applied  to  the  ex¬ 
perimental  unit  at  *  level  lower  then  the  high  level  (1) ,  1.  e. ,  thu  low  level 
ot  a  lector  does  not  imply  the  absence  of  th*  factor.  II  t*  also  assumed  that 
If  a  factor  has  a  rsatdual  affaot  on  any  subsequent  factor,  thla  effect  will 
occur  no  matter  which  level  of  the  faotors  are  applisd. 

Whan  only  two  faotors  art  ordar  dependant,  th*  tffeet  ot  ordering  oen 
be  introduced  ea  another  factor.  If  the  two  faotors  that  are  order  dependent 
are  dsnotad  by  A  end  B  ,  th*  ordering  feotor,  X,  may  be  Introduced  aa 
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follow *.  Let  the  0  level  of  X  Indicate  thet  feotor  A  la  applied  baf or* 
factor  B  ,  and  let  the  1  level  of  X  Indicate  that  factor  A  1*  applied  after 
factor  B,  The  experimental  plan  for  a  2n  factorial  arrangement  where  two  of 
the  n  factor*  are  order  dependent  may  be  deduced  from  a  full  or  fractional 
replicate  of  *  2n*1  arrangement.  Consider,  for  example,  an  experiment  on 
three  two- level  faotor*  A,  B,  end  C,  In  which  A  and  B  are  order  depen¬ 
dent.  Let  X  ba  Introduced  a*  the  order  factor.  An  experimental  plan  may  be 
obtained  by  utilising  the  1/2  replicate  of  a  24  arrangement  with  the  defining 
contraat  l  ■  ABCXg ,  The  treatment  combination*  for  thl*  plan  arat 

A  B  C  X 

0  0  0  0 

0  0  1  1 

0  10  1 

011  0 

10  0  1 

10  1  0 

110  0 

111  1 

Hence,  the  eight  treatment  combination*  of  the  2*  arrangement  In  the  order  In 
which  the  factor*  are  applied  aret  aghgCg,  bgagCj,  biaoog,  apb^C], ,  bobicg. 

*lb0°l'  •ibi°0'  bi»i°i*  U  the  order  faotori  X,  doe*  not  Interact  with  A, 

B,  or  C,  and  all  three-factor  Interaction*  are  negligible,  orthogonal  estimates 
of  Ac  ,  A,  B,  C,  X,  AB,  AC,  and  BO  may  b*  obtained  by  a  »U8lght- 
i.xward  analyst*. 

If  X  InUiuula  with  A ,  3  or  C,  n  3/4  replicate  plan  defined  by  the 
three  defining  contraite 


I  ■  ABCq  -  CXQ  -  ABXg 
I  -  ABCg  M  CXj  -  ABXj 
1  ■  ABOj  -  CXj  ■  ABXg 

will  permit  eetlmete*  of  theee  Interactions,  where,  for  example,  ABO,  denote* 
the  treatment  combination*  for  which  th*  sum  of  the  level*  of  the  factor*  A, 

B  end  C  is  equal  to  1  modulo  2.  The  twelve  treatment  combination*  In 
the  order  In  which  each  factor  la  applied  arai 
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aobo°0‘  bo*o°o'  *obi°i'  bi*o°i'  *ibo°r  bo*i°i*  •ibico'  bi*i°o'  aobo°r 

b^o^,  bgS^CQ,  Tha  estimate*  of  the  effect*  of  Interest  are  correlated 

in  four  aeta  of  threes  pi  ,  CX,  ABCs  £,  B^,  bxs  £,  £h,  a&  6,  x,  A. 

The  method  of  analyst*  for  3/4  replicate  plana  la  presented  In  Addelman  [lj, 

P.LANS  WITH  SETS  Of  TWC  FACTORS  .DEPENDENT  ON  ORDER.  When 
the  reiponie  la  affected  by  the  order  within  eaoh  of  k  aeta  of  two  faotors, 
the  2*  poaalble  ordering*  may  be  accommodated  In  a  2n  factorial  arrange* 
mont  by  introducing  k  ordering  factor*.  The  experimental  plane  will  then  con- 
slat  of  a  fieotlonal  replicate  of  a  2nfk  factorial  arrangement.  Gonalder  a 
2*1  arrangement  with  factors  denoted  by  A,  B,  C  and  D,  where  A  and  B 
are  order  dependant  and  0  and  D  are  order  dependant .  In  this  altuatlon 
there  ere  four  poaalble  ordara,  ABCD,  BAOD,  ABDC,  and  BADO.  Two 
ordering  faotora  may  be  introduced  aa  followat 
Let  the  0  level  of  X  denote  the  order  AB, 

the  1  level  of  X  denote  the  order  BA, 

the  0  level  uf  Y  denote  the  order  CD,  end 

tha  1  level  of  Y  denote  the  otder  DC, 

If  none  of  the  factor*  A,  B,  C  and  D  interact*  with  either  X  or  Y,  and 
If  XY  end  all  three-factor  interaction*  are  negligible,  e  1/4  replicate  of  the 
2°  arrangement  defined  by  I  ■  ABCX0  ■  ABDYq  »  CDXY0  will  yield  orthogonal 

estimate*  of  M  ,  A,  B,  0,  D,  X,  Y,  AB,  AO,  AD,  BO,  BD,  end  CD, 
The  treatment  combination*  of  the  2*  experiment  In  their  improprleto  orders, 
ee  doV'i mined  from  the  defining  oontraat,  ares 


aoVodn 

VuVo 

b0*ld0°0 

•lbl°oHi 

a0b0dl°0 

biWi 

Vl°Odl 

*)bldl°l 

;,o°oa:do 

aobidn°i 

aibodo°i 

bl‘lJldl 

Vodi°i 

aObl°ldl 

albO°ldl 

If  It  1*  expeuted  that  the  treatment  feature  Interact  with  the  order  fac¬ 
tor!,  a  3/8  replicate  of  the  26  arrangement  will  permit  tho  estimation  of 
theee  Interaction*,  a*  well  as  the  mein  effaota  and  two-faotor  interaction* 
that  could  be  eetlmated  with  the  1/4  replicate  of  the  2®  plan.  For  the  Irregu¬ 
lar  fraction  plan  to  yiald  meaningful  estimate*,  the  interaction  XY  and  all 
three-factor  interactions  must  be  negligible.  The  three  l/B  replicates  of 
the  2®  system  aro  defined  by 
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I  -  ABCDXj  -  ABYQ  -  CDXYj  -  ACDYg  -  BXYj  -  BCD0  -  AXj , 

I  -  ABCDX1  -  ABY0  -  CDXY1  -  ACDYj  -  BXY0  •:  BCDj  -  AXQ, 

and  I  ■  ABCDXj  ■  ABYl  -  ODXYQ  -  ACDYj  -  BXYQ  ■  BCDQ  -  AX1 , 

respectively,  The  twenty-four  treatment  combinations  of  the  2  4  experiment 
in  their  appropriate  orders  of  appltoatton  are; 


bo*ococlo 

aobo°odi 

boaodo°o 

b0*0°ldl 

*ObO°ldO 

bOaOdl°l 

* 

bl*0dl°0 

a0bld0°0 

bl*0°0dl 

VoVi 

*Obldl°l 

blaO°ldO 

• 

•ibo°odo 

b0*ldl°0 

•ibo°odo 

•lbOdX°l 

bOaldO°l 

•lbO°ldl 

alblo0dl 

bl*l°OdO 

°lbldl°0 

•lbl°ldO 

bl*l°idl 

•lbldO°l 

IT"  least  squares  estimates  of  the  estimable  parameters  are  partially  eonf<*.ll:Jed 
in  sets  as^  follows; 

/L  and  fo:  'A,  X,  end  fiV;  S,  AY,  >mcl  JiJ;  6  and  bI);  6  and  fib;  Y, 
fo.  and  fik;  K&,  &,  end  dV;  OY,  AD,  and  dx, 


k  OTHER  FAHTOhL..  | 
factorial  expori-  ; 
munt  Is  dependent  on  the  position  of  one  faotor  relative  to  It  of;  tho  remaining 
n-1  factors,  !tn  tha  sequence  of  application,  the  2*  possible  orders  mciy  oo 
accommodated  by  the  Introduction  oi  k  order  factor*.  This  type  of  plan  oan 
be  Uluetrated  by  an  experiment  on  four  two-level  factors,  A,  B,  0,  and  D,  . 
where  A  le  order  dependent  with  B,  0,  and  D.  Tho  three  order  faatore 
X,  Y,  end  Z  may  be  defined  as  follows , 

Lot  the  0  level  oi  X  denote  the  ordor  AB, 

the  1  level  of  X  denote  the  order  BA,  i 


Whan  tha  reaponee  to  a  treatment  combination  of  e  2' 
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the 

0 

level  of 

Y 

denote  the  order 

AC, 

the 

1 

level  of 

Y 

denote  the  order 

CA, 

the 

0 

level  of 

Z 

denote  the  order 

AD,  and 

the 

1 

level  of 

Z 

denote  the  order  DA. 

The  eight  possible  orderings  ere  ABCD,  BACD,  CABD,  BCAD,  DABC,  BDAG, 
CDAB,  end  BCDA , 

?  The  2  4  arrangement  with  three  order  factors  may  be  considered  to  be 
a  2  arrangement.  A  1/8  replicate  o.  the  2 7  arrangement  defined  by 
I  •  ABXYZq  -  BCXY0  -  ABCZ0  -  ABDXg  -  BDYZg  -  ACDYg  -  CDXZ0  oonslats  of 

the  following  16  treatment  combinations  with  each  factor  applied  in  the  appro¬ 
priate  order. 


<«  b„c  d 

0  0  0  0 

b0dl*0cl 

Vlb0°0 

Vi°idi 

•obi°idi 

bld0a0°D 

do“lbl°l 

bi*i°od: 

bo°ottodi 

°l*lbOdl 

bo°odoB 

ViVi 

bi°i*odo 

Vibido 

bl°ldl* 

If  all  Interactions  involving  at  least  one  order  factor  and  all  thiee-faotor  and 
hlgncr  order  interactions  are  negligible,  the  above  plan  permits  orthogonal 
estimation  of  A,  B,  0,  D,  AB,  AC,  AD,  BO,  BD,  CD,  X,  Y,  and 
Z. 

With  the  usrr.p  aiuumpucns  "s  above,  a  plan  may  be  constructed  for 
the  25  arrangement  on  factots  A,  a,  C,  D,  and  E  where  factor  A  is  order 
Hepnn.lwnt  with  factors  B,  C,  and  D.  This  plan  is  derived  from  .ho  *raat- 
meni  combinations  of  the  i/8  replicate  of  a  28  arrangement  defined  by 
I  -  ABGnEg  -  A  BOXY g  -  DEXYj  -  ADXZ0  •  BCEXZq  -  BODYZq  -  AEYZq,  whore 
X,  Y,  end  Z  are  the  order  factors •  If  It  is  important  to  estimate  ‘ML  the  two- 
factor  Interactions,  a  3/16  repltcate  of  the  2®  would  provide  those  esti¬ 
mates,  some  of  which  are  partially  confounded  in  seta  of  two  or  three.  Thu 
generator*  of  the  defining  oontrast  of  the  three  i/16  replicates  which  define 
such  a  plan  are  ABODE,  ABXYZ,  ACX  and  BEY. 

PLANS.  WITH  XHltEE  FACTORS  DEPENDENT.  ON  ORDER  .  In  thl*  section 
wo  consider  plans  for  experiments  in  whloh  ail  six  permutations  of  the  orderings 
of  three  factor*  mty  Influence  the  raeponoe  to  a  treatment  combination.  If  the 
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three  factors  that  are  order  dependant  are  denoted  by  A,  B,  and  C,  the  elx 
orders  tn  which  these  three  factors  occur  are  ABO,  ACB,  BAG,  BCA,  CAB,  and 
GOA.  When  there  are  more  than  three  factors  Influencing  the  reeponoe,  the 
factors  other  than  those  denoted  by  A,  B,  and  0  may  occur  any  place  in  the 
sequence  without  affecting  the  response.  The  sir  orders  may  be  accommodated 
in  the  experimental  plan  by  Introducing  three  order  faotora  X,  Y,  and  Z  as 
follows; 

Let  the  0  level  ol  X  denote  the  order  AB, 

the  1  level  of  X  denote  the  order  BA, 

the  0  level  of  Y  denote  the  order  AC, 

the  1  level  of  Y  denote  the  order  CA, 

the  0.  level  of  Z  denote  the  order  BC,  and 

the  1  level  of  Z  denote  the  order  CB. 

The  order  factors  can  be  treated  like  three  two- level  factors.  Of  the  2"  ■  B 
treatment  combinations  of  the  three  order  faotora,  two  combinations ,  010  and 
101,  give  Impossible  orders.  Whenever  these  orders  occur  In  the  plan  they  must  be 
changed.  If  the  experiment  consists  of  three  factors  A,  B,  and  C  which  are 
all  order  dependent,  an  experimental  plan  may  be  obtained  from  the  treatment 
combinations  that  comprise  the  1/4  replicate  of  a  2<>  arrangement  defined  by 


I  -  ABXY0  -  AOXZq 

"  BGYZq  , 

namely: 

A 

B 

O 

X 

Y 

z 

0 

0 

0 

0 

0 

0 

0 

0 

0 

i 

1 

1 

0 

0 

1 

0 

0 

1 

0 

0 

1 

1 

1 

0 

■  0 

1 

0 

0 

1 

0* 

0 

1 

0 

1 

0 

1  ■' 

n 

1 

1 

1) 

1 

1 

0 

1 

1 

1 

0 

0 

1 

Cl 

0 

0 

1 

1 

1 

0 

0 

1 

0 

0 

1 

u 

1 

0 

). 

0* 

0 

1 

1 

0 

i* 

1 

1 

0 

0 

0 

1 

1 

1 

0 

1 

1 

0 

1 

i 

1 

u 

0 

0 

1 

1 

1 

1 

1 

1 
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The  four  treatment  comblnulsn*  marked  by  an  asterisk  lead  to  Impoailble 
orderings  and  hence,  the  combination  of  the  X,  Y,  Z  levels  must  be  changed 
in  these  treatment  oomblnatlons.  This  can  simply  be  accomplished  by  reversing 
the  levels  of  Z  in  the  four  treatment  oomblnatlons.  The  resulting  treatment 
combinations  of  the  23  arrangement  with  appropriate  orderings  of  the  factor 
levels  are: 


aofco°o 

°0a0bl 

°0*lb0 

ai°cb 

°oboao 

bla0°0 

b0*l°0 

bl°0a 

Vib0 

ClSObl 

cllalb0 

•lbl° 

Vla0 

bldOcl 

Vi°i 

°lbla 

This  plan  permits  orthogonal  estimates  of  all  treatment  effects,  the 
order  effoats  X  and  V  each  being  partially  confounded  with  Z  ,  The  esti¬ 
mates  of  the  treatment  effects  are  simply! 

A  -  iW  -  -  iO],  6  -  L[o]  ,  AB  -  |[ab]  ,  AO  -  1[ac]  , 

BC  -  g[Bcl,  ABO  -  g[AB0], 

where  [ A  «  (sum  of  neatment  oomblnatlons  with  'auor  A  at  l  level- 
.  sum  of  treatm>»nt  combinations  with  factor  A  at  0  level), 

I  AB|  m  (sum  of  treatment  combinations  whose  expectation*  con¬ 
tain  AB  positively  -  aum  of  treatment  oomblnatlons  whoa ; 
expectation*  contain  AB  negatively), 
and  so  on. 

The  estimates  of  the  older  effects  are  given  by 
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The  above  estimates  art  valid  if  ail  Interactions  involving  at  leaat  one 
order  factor  are  negligible.  If  the  lw6-taotcr  interaction*  involvtno  one  order 
factor  are  not  negligible,  the  sixteen  treatment  combinations  of  the  previous 
plan  are  not  adequate ,  Whan  all  twe-faotor  Interactions  among  the  order  fac¬ 
tors  and  all  throo-faotor  and  higher  o'rdarLrfterj&atlokia; involving  at  least  ona  order 
factor. are  .negligible,  a' 3/Srepllcete.  of.'the  2*.arrdngamertt  will  perm/t  estimates 
of  the  remaining  parameter* .  From  the  three  1/8  replicates  defined  by 

I  ■  XYq  ■  V7,0  -  XZ0  -  ABCj  -  ABCXtTi  -  ABCYZj  -  ABOXZj  , 

I  “  XYj  -  YZQ  -  XZ:  -  ABCq  -  ABCXYj  «  ABOYZq  -  ABCXZj  ,  end 

I  -  XYq  -  YZ^  -  XZj  •*  ABCq  -  ABCXY0  -  ABCYZj  -  ABOXZ]l 

the  treatment  combinations  of  the  2  arrangement  in  tho  appropriate  sequences 
of  application  may  ba  deduced,  Some  care  was  taken  in  selecting  the  three 
1/C  replicates  so  that  no  treatment  combination  that  resulted  from  the  selection 
of  the  defining  contrasts  would  involve  impossible  orderings.  The  24  treat¬ 
ment  combinations  in  their  appropriate  order*  are; 


•obOcl 

°ibla0 

bo*o°o 

°oaobo 

bo°oao 

■o°Ob 

•obi°o 

°0bia0 

bl*0°l 

°laObt 

bi°i*o 

Vlb 

alb0°0 

°0b0al 

b0ala‘ 

°lalb0 

buciai 

Vlbl 

albl°l 

°lblal 

blala0 

°Oalbl 

bl°Oal 

\  «0- 

Tnt-j  estimates  given  by 

3  1  I  T  I  ,  where  T  denotes  the  sum  of  all  trestin.  n* 

'I  .  I 

1  3J  L  fABCl  J  combinations. 


r  a 

A 

_  J.  . 

'  3 

1  ‘ 
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rsi 

3  l” 
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l  AC  J 

1  3 

[ac! 

•  -L  • 

'  3 

l1 

Lei 

.  AB. 

32 

.  1 

3 

M 

i 

a-' 

Ano 


JL 

32 


394  Design  of  Experiments 


A 

X 

» .i 

w 

A 

AX 

2  -1 

1 

W 

A 

Y 

„  _L 

16 

-l  2  -1 

M 

A 

AY 

‘"f? 

-1  2 

-1 

N 

A 

l 

1  -1  2 

« 

.z3 1 

* 

A 

AZ 

1.  -i 

i 

-  A 

BX 

'2-1  1  ' 

L&rf 

A  • 

cx 

'  -1 

ll 

A 

BY 

M 

16 

-1  2.  -1 

A 

CY 

”uT 

-1  2 

- 1. 

N 

A 

i  BZ 

L 

1-1  2 

P 

N 

> 

<?Z 

1  -.1 

2 

N 

REFERENCE 

Addelman,  S.  1961.  Irregular  freotlona  of  the  2n  factorial  experiment!, 
Teohnometrio*  12i4?9-496. 


2P  FACTORIAL  EXPERIMENTS  WITH  THE  FACTORS 
APPLIED  SEQUENTIALLY 


R.  R.  Prairie  and  W.  J,  Zimmer 
Sandia  Corporation 

ABSTRACT  ,  In  the  application  of  factorial  experiment*  In  induetry,  It 
1*  often  neceesaiy  to  apply  the  factors  sequentially.  For  ZP  experiments 
in  which  (a)  multiple  testing  on  each  experimental  unit  is  possible  and  not 
degrading,  (b)  the  effects  of  the  factors  are  permanent,  and  (c)  the  low 
level  of  a  factor  is  the  absence  of  that  factor,  a  design  and  an  analysis  are 
presented  which  allow  the  estimation  and  testing  of  order  effects  as  well 
as  the  usual  main  and  interaction  effects  of  the  factor*, 

INTRODUCTION.  In  the  usual  application  of  factorial  experiments,  the 
factor  levels  are  applied  simultaneously  to  the  units  and  a  response  is 
recorded  from  each  unit.  However,  when  the  factors  are  environments 
(e.  g. ,  vibration  and  mechanical  shook),  it  is  often  impossible  to  apply  the 
fi-'tor  levels  simultaneously,  In  this  situation,  the  factor  levels  are 
applied  in  sequence, 

When  the  factor  levels  must  be  applied  sequentially,  the  or0.:<<  of 
application  become*  a  matter  of  importance,  It  is  possible  that  tho  final 
peilormance  of  a  unit  that  has  been  subjected  to  two  factors  mtiy  be  very 
sensitive  to  the  order  in  which  the  factors  are  applied. 
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In  this  paper  2  experiments  are  considered  where: 

1,  F.ach  unit  may  be  tested  (p  +  1)  times  (where  p  is  the  number 
oi  iactors)  without  the  tasting  itself  having  a  degrading 
effect  on  the  unit, 

2,  The  high  level  of  the  factor  is  the  application  of  the  factor 
and  the  low  level  of  the  factor  means  the  factor  la  not  applied, 
and 

3,  The  effects  of  the  factors  are  permanent. 

Thus,  the  magnitude  of  the  response  from  a  specific  test  will  be  affaeted  by 
the  factors  applied  to  tho  unit  up  through  that  teat  plus  the  specific  order 
of  application  of  those  factors.  A  deeign  [  railed  the  Factor  Sequencing 
Design  (FSD)  ]  and  an  analysis  are  presented,  which  allow  the  estimation  of 
these  order  effects  as  well  as  the  usual  main  and  interaction  effects  of  the 
factors, 

LfwSION  AND  MODEL,  Each  factor  [  C  *  1,  , ,  , ,  p]  has  a  high  level 

t  bee  been  applied]  and  a  low  'evel  [  hae  not  been  applied.  Thus,  when 

a  unit  has  the  high  level  of  tppiied  lo  It,  we  shall  say  that  it  has  received 
the  V  factor  or  that  the  factor  has  bean  appliedj  Sines  the  affects  o» 
the  factors  are  assumed  to  be  permanent,  a  unit  which  hae  received  tin.* 

Vf  factor  at  some  point  must  be  considered  from  that  point  on  as  having 
tne  high  laval  of  Vj  .  Further,  each  unit  will  eventually  receive,  in  '..une 
order,  each  of  the  p  factors.* 


*1*  The  remainder  of  this  paper  has  been  reproduced  photographically  from 
the  author's  copy. 
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Th*  type  of  dailgn  considered  la  om  whan  r  unit*  an  aubjaotad  to  aaoh  of 
tho  pi  poiaibla  order*  of  tha  p  fast  ant  V^,  Vg,  ...,  Vp.  ®ba  p  factor*  an 
applied  In  aaquanc*  and  aaoh  unit  1*  taatad  (p  +  1)  tinea,  one*  prior  to  tha 
Application  of  any  factor  and  once  following  tha  application  of  aaoh  of  tha 
p  factor*.  Thu*  a  notation  for  the  napon**  on  the  Jth  teat  (j  «  1, 
p  +  1)  of  the  itn  unit  (l  ■  1,  ...,  r)  within  or,  in  othor  word*,  eventually 
aubjactad  to  tha  kth  p-vay  order  (fc  ■  1,  pt)  lot 

MV  •••' v4j. 

whan  a,  ,  Indloete  the  J-l  factors  which  ham  bean  applied  up  through 

th  ^  J"1 

tha  J  toat  and  tha  order  of, application.  If  J  ■  l, 

i.e.,  no  faotora  havo  bom  applied,  tha  napoaae  la  aywboliaad  by  *ilk^)’ 
Although  thia  notation  la  ndundant,  it  waa  selected  because  of  it*  convenience 
when  daaorlblng  tho  analy*l»>  An  axaapla  of  tha  daelgn  la  given  for  a  2^ 
•rperimant  in  Sable  1. 
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TASIfl  I 

Saaiw  of  *  8s  W)  in  lix  Diffarant  Ordari 

Easiai  liiS-L.  Mu  .lilt  8.,  Zu&u  ,AU.  Ua&ai  it 

Ordar  It 


Unit  1 

• 

Nona 

i 

XUA^ 

• 

a 

JW*i) 

a 

va 

a 

’WVa* 

a 

V3 

a 

xi»tX^ii»*j^ 

a 

• 

Unit  r 

1 

Mona 

• 

xrllu> 

a 

VX 

a 

a 

VB 

a 

Xrjl^l*a^ 

a 

*3 

a 

WviV 

Ordar  Bi 

' 

Unit  X 

a 

t 

Mona 

• 

a 

x^U) 

• 

a 

VX 

a 

a 

5W,P 

a 

V3 

a 

xA3bKV 

a 

V8 

a 

xi4a^iV») 

a 

Unit  r 

t 

> 

Mona 

• 

a 

a 

Vx 

a 

X*a<“i> 

a 

a 

V3 

1 

a 

Xrja^X*S> 

a 

a 

a 

va 

a 

a 

• 

xrta^xBj,a) 

a 

a 

Ordar  61 

Unit  x 

a 

Mona 

a 

a 

*u«U> 

a 

a 

V3 

a 

a 

X1*<*J> 

a 

a 

VB 

X136(ajV 

a 

V1 

a 

W“jVJ 

a 

Unit  r 

a 

Nona 

a 

Wl> 

a 

V3 

Xri6**3* 

a 

VB 

a 

XrS6^,J#8^ 

a 

Vi 

X»W*3*lal) 
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Th*  nodii  that  la  uaed  to  mnitst  a  tost  tor  the  2P  WO  la 
XUk(V  ®4j.x)  "  B  *  ul(k)  +  <0^,  ^  *  *ijk 

1  •  1,  2,  . r 
J  “  1«  Si  >H|  f 
k  ■  1)  2j  ••••  Jl 

wh are 

xljk/a4  ‘  aJ  )■***•  KifHH  obtained  on  tin  Jth  t*»t  fircn 
\  1  imXl  *h 

tin  iw‘  unit  eventually  auhjeotad  to  tho 

kth  p-xay  order  of  tin  footer#  where 

•j  *  • *  *»  •*  i*  a  particular  ordering 
1  J-l 

of  th*  1-1  factor*  applied  up  through  tho 
1th  toat. 

a  ■  (antral  naan. 

ui(k)  “  of  ****  unit  *v»UaJUy  eubjeoted  to 

ky“  p-wajr  order  of  the  p  faotora. 

Ku.  „  ■  affoet  of  the  applloetltei  of  J-l  faotora  In 

*1  *1  1 

J  tho  a pool fie  order  a.  ,  a*  ,,  for 

1  J-l 

J  n  X,  tho  ayahol  uaai  la 

*ljk  “  r*nd0*  *rr9r  uaoolatad  with  Jth  taat  on  the 
1th  unit  eventually  auhjaotad  to  tht  1th 


order  of  p  faotora. 
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Iter  the  a3  PSD  .on*  .ptoltt*  *  «»•  fora  of  ®04,X  u#l 

xm(i)  -  »  +  «i(i)  4  *(«  4  “Ui 

" *  *  *wd  +  S *  ,12X 

HsM1  - B  +  U1(1)  4  *Va  +  *131 
Wws*  “  * 4  ua(i)  4  Svj +  *l4x 


XU3(1)  *  *  +  ^(S)  4  *U)  4  *JM 

XU3(«8>  -  »  ♦  «x(3)  +  <«9  *  "M3 

WVl>  “  *  4  ttl(3)  4  S°1  *  *133 


W#a\V 

• 

-B  +  U1(3)  4|»aV3  +  *143 

* 

% 

I 

xug(i) 

-  »  +  «l«i) 

1 

+  *(1)  4  *U6 

WV 

•  *  +  ui(6) 

*  *a3  +  #12b 

wv** 

- *  +  V«) 

+  ^3oa  4  *X36 

xl^a302Bl^ 

•  B  +  ul(6) 

*Svl**« 

It  vlll  b.  M.unid  that 


uiW  -  NJD<0,  ffj),  «1Jk  -  KD(0»  aj) 


Deiign  o l  Experiment* 


'■.01 


and  that  each  fc,  „  ia  tha.iua  o  t  tvo  parametar*,  on*  aaiooiatad 

1  V  1 

vlth  tha  factor*  Involved  and  ena  vith  tha  ordar  of  tha  fact ora,  l.e.« 

^ ft i  .  ■  1 1 ,  ft.  ^Bil  i  mi  ft .  i  Q,  ,  1 1 •  ,  8j 

T.  >1  1  4j-l  *1 

whart  t)_  .  la  tha  parameter  aaioeiatad  vlth  tha  J-l  faetore 

U|,  ...»  are  tha  ...»  4^  ordarad  from  tha  amallaat  through  tha 
lursaat)  and  8.  „  la  tha  parameter  uaoolatad  vlth  tha  apaolflo 

1  Vl 

ordar  of  application  of  tha  J-l  faotora. 


It  la  alao  aaaumad  that  tha  ('a  aun  to  taro. 
From  Squat Ion  1  it  oan  ha  aaan  that 


cov 


•  ••*  aj  Xi'.'ui/a.  #  .*•<  a.  \ 

>1/  1  J  *  \  >1/J 

l  r-  1',  3  -  J',  k  •  k* 

<> 

t  •  l1/  3  *  3'/  h  •  h1 

(a) 

0 , 

otharvlaa. 

Anaiyaia 

Binoo  tha  modal  uiuatd  ii  linear,  tha  propeaad  analyila  la  baaed  on  tha  n.*thod 
of  Laaat  aqutuaa.  From  Equation  1  It  la  apparent  that  l(X)  ■  A*|#  vhara  X  la 
tha  r(p  +  l)p!  x  1  column  vaotor  of  obaervetiona,  A*  la  tha  r(p  *  l)pl  x 


1*SlxA^ 


daalgn  matrix  and  (4 


i-  tha  1  ♦  j£j  ^yr 


x  1  column 
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vootor  of  penuieter* .  Now  the  rank  of  A*  le  y  "^yyr  vWL«h  1*  on*  i*M 
then  the  number  of  par  meter  1.  Sinoe  th*  tmaSr  at  eetimabl#  parameter*  It 
aquul  to  the  rank  of  A*,  e  reporitaeteriietlon  must  he  performed.  To  aoocnpllah 
thl*  a  nev  vector  C +  le  defined.  Then  S(X)  r  A*M<+,  where 

t+'  -  £a  +  <(1).  ^  -  <(i)»  •••>  «axo8,  ...,  «s  ’  <(l)' 

■•-sw-vH’ 

le  of  order  ^  *  M  le  the  matrix  of  repereemterieation,  end  A*M  hae 

; 

Xt  ie  not  yet  poeeihle  to  obtain  th*  beet  eet'lmat*  of  (+l  from 
r  •  [(A*M)'(A*K)]“l(A«M)'X 

beoauee  the  Oauie-Karkov  (a-M)  theorem  le  not  applicable,  ee  the  X'e  ere  not 
indopendently  Oietnbutea  [ee*  Equstlon  8) . 

In  matrix  notation  Equation  8  1*  written! 

Var  (X)  -  »*I  +  oj  die*  {u}  (3) 

where  I  le  the  identity  matrix  of  order  (p  +  l)rpi  end  dleg  Ju}  le  a  equate 
Ataconal  matrix  of  ordar  (p  +  l)rpl  for  vhloh  each  diagonal  element  le  th* 
eubmatrlx  U,  n  equate  matrix  of  ordar  (p  +  l)  all  of  whose  element*  are  1. 
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To  aatiify  tna  oonditlota  of  tha  O-M  thaoraa,  a  linaw  txanaforaatlon, 

Z  -  BX 

oan  ba  found  »uoh  that  var  (fc)  ■  #jl,  whara 

Z>  “  [*111'  *191'  hfif  *911'  •'*'  *9pl'  *rlpl'  ••••  **»l] 

flinea 

vac  (z)  •  Btvar  (X)]B‘  -  b|oJi  +  oj  dla«  |u|]b’  ,  (4) 

a  matrix,  B,  vhiah  pouaaaaa  tha  proportiai  SB1  -  X  and  ii[dla*  jutj>'  ■  0, 
vin  yiald  var  (Z)  ■  ojl,  A  portlouler  matrix  that  yialda  tha  daalrad  raault 
la  a  rppl  x  r(p  +  l)pl  matrix  B  vhara 
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and  v'nara 


*  (Z)  -  »[»  (X)).  ■  , 

and  slur*  the  firat  aoluao  © t  BA*H  la  all  aaroa,  aat 
S  (Z)  -  A« 


where  A  la  a  jippt  x  “teix  la  BA^M  with  tha  firat  oolwm, 

which  la  a  colwan  of  aaroa,  dalatad  and  I  1*  tha  parameter  net  or  <+  raduoad 


by  delating  tha  firat  elaaent 


* 
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Since  Z  linear  transformation  of  normally  distributed  variables,  the 

Z's  art  normally  distributed.  As  ver  (z)  »  c^X,  the  Z's  satisfy  t»m  conditions 
of  tbs  C-M  theorem.  Thus,  the  bast  ilnesr  unbiased  estimate  of  {  la 

t  -  (A'AJ-Vz  .  (5) 

I 

An  analysis  of  varlsncs  associated  with  l^uatlon  $  eaa  ba  given.  This  analysis 
of  vaxlanee  appears  la  Sable  IX. 

SABI*  XX 

PrsUaina./  Analyst*  of  Variance  Including  All  Parameters 


Source  of 
Variation 

-,(wL- 

Sreatmsnts 
{ factors 
and 

or  dire ) 


Degress  of  freedom 

_ (3a) _ 


Bun  of  Baueres 


f  Sac 

ML 


r  - 1)‘ 


-  t'(A(A'A)"1A')z 


p-2 

Jirror  ^  {jT^XTT  '  1)!  “  l*  +  a(r  *’  *)5!  “*’(*“  A(A'A)''■l■A,)  Z 


Total 


rppl 


*  *  «'* 


By  neglecting  order,  a  nav  ^urasatar  vector  Is  generated.  Denote  this  nrv 
ve^t  jt  as  t)  .  for  example,  v-th  a  2^  VBD 


n'  “  [\  '  TKi)*  \  ‘  ^D'^sj  "  TU)*%oa  ”n 

\tt3  ‘  T^1>'T'8 a*}  "  T^1>'r>aiaa<*3  ‘  n(1)]  ‘ 


(i) 
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The  host  estimate  o f  T)  la 

n  -  (a^)*1/^  ,  (6) 

whore  is  jrppl  x  tP  -  l]  matrix  and  la  BA^M^  vlth  the  first  column  deleted, 

whore  A*  is  the  design  matrix  associated  vlth  and  la  tha  matrix  of 
reparomateriaation  associated  vlth  1}  • 

The  corresponding  aaalyr  la  of  varlsnca  la  given  in  Table  HI. 

TABU)  XXI 

Preliminary  AnaJyala  of  Variance 
Disregarding  Order  Parameters 


Intel  rpp!  z'fc 


Then,  by  aeons  of  the  principle  of  conditional  error,  the  anme  of  squares,  due 
to  the  factorial  effects,  the  order  effects,  end  the  a:rror,  can  be  obtains'*  tu 
shown  In  Table  XV .  ‘ 
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table  iv 

i  ■■ 

Analysis  of  Vorlonoo  for  Footorlsl  and  Ordsr  Effects 


Factor* 

®p-i  *'[VWiAa]* 

Ordoro 

1 

i 

§  (?)(“  ■ 

d‘[A(A,A)*^A'  -  A1(^A1)"1Ajl 

Error 

9*8 

fa  ip  ■  t)i 

t»(»  -  i)l  -  l)  ♦  8(r  -  l)pt 

I'[l  -  A(A'A)"lA']a 

Total 

*») 

It 

Zt  aoo  too  shorn  that  ttoo  sues  of  KWr«i  are  lodevendsntly  distributed  m 

X*iPt  and  tho  sub  of  square*  duo  to  order  to  sloaa  of  factorial  offooti.  Thus, 

tin  slcnlfloonoa  of  tho  order  offosto  sob  too  tootod  by  tho  us; tX 

F-v»it  and  it  ordor  offooti  oxo  Mglitlblo  tho  footoriol  offooto  oob  oloo  too  tested 
toy  vln  F-toit. 

Zn  tho  fluo  vhsr*  ordor  offiot  If  found  Mdllitlbia,  any  given  footoriol  offset*  aoy  »y, 
voiUi'  too  estimated  by 

^  •  4  9|  ■  fl'to  , 

vhere  ^1  to  (Ivan  toy  Equation  6  and  d'  to  tho  victor  Vhioh  ftvoa  tho  appropriate 
llnoar  combination  soon*  tho  parameters.  Tho  sum  of  square i  corresponding  to 
that  offoot  to  (O'l)a/o'0  . 
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Xn  tha  cuo  where  the  order  of  application  of  the  faotor*  hue  a  significant 
affect,  ona  would  presumably  Investigate  different  contrast*  among  parameters 
of  £,  an  estimated  by  Equation  5,  to  determine  which  are  contributing  to  the 
order  effect .  This  can  bo  dona  by  calculating  expresalons  of  the  form  g 
'/hero  gla  a  vector  defining  a  specific  order  contrast  of  Interest.  If  order 
In  nlgnificant,  one  should  bs  careful  about  inference!  concerning  factorial 
effects. 

It  lo  interesting  to  noto  that  whon  pit,  the  number  of  units  needed  for  the 
PSD  (pi,  or  24  when  p  ■  4)  is  greater  than  the-  number  (2s,  or  16  when  p  ■  4) 
neodsd  for  the  usual  2s  factorial.  In  this  case,  however,  tha  advantage  of 
the  PSD  over  the  usual  factorial  is  tha  information  obtained  on  tha  order 
effects,  and  th*  increased  efficiency  of  estimation  due  to  tha  removal  of  unit 
variation  frets  tha  error.  Furthermore,  it  ie  believed  that  a  fractional 
replicate  of  the  PSD,  using  fewer  unite  than  the  ueual  factorial,  would  etlxl 
provide  estimate*  of  both  the  factorial  effeota  and  tha  order  affaota.  This 
or.pact  of  the  IDS  X*  presently  bell*  investigated. 

Also,  far  the  case  where  there  is  no  order  effeot,  it  le  possible  to  construct 
a  design  which  requires  only  p  unite  end  for  which  It  is  possible  to  estimate 
all  2S  ■  1  factorial  effects.  This  design  is  simlxar  to  a  Latin  square,  vher-. 
each  unit  la  tested  p  +  1  times. 

The  general  FSD  technique  described  above  will  uov  be  Illustrated  by  two 
a 

examples  of  a  2  experiment. 
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Rxuqple  1  of  ft  S3  WD 

Coneidar  ft  83  WO  who**  dailga  ft&4  reeult*  in  them  la  Tftble  V. 

TABU  V 

Neoulte  Frcffl  One  Replication  of  ft  S* 


1-W»v  Order  2I0SSS 

fleet  1 

yen  tor 

mu 

Factor 

Tftet  1 

2l gjat 

Teet  4 

Order  1 

Unit  1 

Nona 

$6.8.18 

V1 

56.579 

V8 

98.661 

V3 

31.319 

Order  8 

Unit  1 

Ronn 

33.500 

V1 

57.461 

V3 

5T.475 

V8 

50.396 

Or iW"'  3 

Unit  1 

None 

58.515 

V8 

56.383 

V1 

68.083 

V3 

61.673 

Order  4 

Unit  1 

None 

56.583 

V8 

56.9a4 

V3 

96.ni 

V1 

68.083 

Ordar  ‘i 

um  1 

None 

94. BIT 

V3 

55.914 

Vi 

53«9r4 

V2 

49.734 

Order  6 

Unit  1 

None 

56.034 

V3 

57.893 

V8 

55.440 

V1 

60.863 

The  nodal 

need  to  repreient  any  reiponae  li  given  by  Rr,u*U«a  It 

^IJkf^]/ 

...,  a.  \«»  + 
*4-l/ 

ut(k)  * 

>  ...» 

1 

ai  + 

M 

•ljk 
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vharo  1  ■  J  ■  1,  .2,  3,  bj  b  ■  1,  2,  3*  b,  5 ,  6. 


How, 


and 


~4e  (56.258  "  >6*579) 

V2 


(56.858  4  56.579  -  2 [58.661)), 


it  -  BX- 


■^ss  (56.258  +  56.579  +  52.66l  -  3131*315)) 


~^ba  (56.03b  4  57*895  4  55-440  -  3(62.863)) 


12'  .  [-0  ri2T,  3-063,  3.335,  -1.387,  -0.8ia,  5-556,  1.550,  -3.759,  -2.355, 
-0.241,  0.52b,  -b.803,  -1.200,  0.891,  b.235,  -1.316,  1.245,  -5»5b8)  . 


r»  ■ 


Than  the  liast  aotlaata  oft 

«'  “  [\  "  «(!)»  ‘  <(1)'  *”*  KW*  "  <(1)' 

la  [aaa  Equation  5) 

$  -  (a'a)"1*1!!  (7) 
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and 


•  (l.l4l,  -0.9R5#  1.779,  -3.18T,  4.l4l,  1.565,  -0.a08, 

••1.105,  -0.635,  -4.553,  -5.514,  3*791,  4.669,  -4.422,  6.788)  . 

An  analysis  of  varlanae  associated  with  Equation  7  and  corresponding  to  that 
of  Table  IX  Is  given  In  Teble  VX. 


TABUS  VX 

Prsllalnsry  Analysis  of  Varlanae  for 
a  8^  PSD  Xnoludlng  All  Poreasters 


. BY . .  ..  U 

TreatoentB  15 

(foot ore  and  orders) 

Error  3 

Total  18 


_ fit _ _ 

81  |a(A*A)"1A,J  B  -  151.90 

S'  [x  •  A(A'A)"^A']e  ■  p  ,88 
P.'E  u  154.18 


By  neglecting  order,  an  analysis  of  varlanae  corresponding  to  th*.t  .of  Teble  XXX 
cen  oe  carried  out,  as  shown.  In  Table  vn. 
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TABU  VH 

PrcUalnary  Analyala  of  Variance  for 
«t^nD  Bderegardlng  Order  Paraaetara 


. — -g - 

Factorial  Effect* 

2£ 

7 

_ -M _ 

E,(Ax(AiA1)*iA;|a  -  36.98 

Raaldual 

11 

z'|i  •  VWlAi|B" 

Total 

18 

t'E  >  IjU.lB 

In  Table  VX3I  tha  aulyala  of  vattanoe  oorreapoodln*  to  Table  XV  for  factorial 
and  order  affaota  la  given. 


TABU  vrn 

Analyala  of  Variance  for  Factorial 
•ad  Order  Iffaota  fra*  a?  FHS 


sv 

U' actor* 

w 

7 

_ 

-  36.99 

OrJf r» 

8 

8'[a(A’a)“V  -  A1<A^XjL)"a,A^]»  •  9<t,91 

ilrror 

3 

e’[i  -  a(a'a)"aa']  ■  a. 86 

18 


Total 


t'l 
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Teats  of  significance  for  these  particular  order  effect*  could  be  perfomed 
toy  the  "7"  teat  or,  equivalently,  by  the  usual  "t"  test  vhera, 

t .  . ,  nmmaA  iffirt _ 

(nf )  standard  error  of  estimate 

and  n#  is  the  number  of  degrees  of  freedom  associated  vith  error.  The  testa  are 

Vs  Vl'»<3)  ’  ‘  ■8'’“ 

Vs Vs“(a)  *  ■°'kU' 

It  iu  apparent  that  there  is  a  large  effect  resulting  from  V^Vj  versus  VgV^. 

With  th«  large  order  effect,  the  Interpretation  of  the  factorial  effaot*  is 
difficult. 


Sxeaple  2  of  a  2*  FSD 

Consider  a  £  FSD,  the  design  end  results  gf  which  are  shorn  in  Tabic  DC. 


Design  of  Experiment* 


415 


TAILS  ZX 

Jitiultc  Fra*  One  Replication  of  e  S'*  TED 


aaiLMtt 

Feat or 

stall 

Factor 

Teat  fl 

Teat  3 

Teet  4 

Order  1 

Unit  1 

Non* 

9.819 

V1 

IS. 061 

va 

14.930 

V3 

17.330 

Order  8 

Unit  1 

Non* 

11.095 

V1 

13*964 

V3 

13.738 

V8 

14.033 

Order  3 

Unit  1 

Non* 

9.541 

V8 

8.517 

vi 

13.813 

V3 

13.9M 

Order  4 

Unit  1 

Non* 

9.104 

va 

11.484 

V3 

9.109 

V1 

14.133 

Ordi.r  3 

Unit  1 

Non* 

9.045 

V3 

9.173 

V1 

14.836 

V8 

14.701 

Order  b 

Unit  1 

Non* 

18.596 

V3 

9.886 

V8 

10.391 

V1 

19.806 

1‘be  tranefometion 
Z  .  XX 

yields 

Z'  -  (-4.131*  -8.894,  -3.8A,  -3.443,  -1.796,  0.800,  O.T>4,  -5.049,  -8.438, 
•1.663,  0.966,  -3.684,  -0.091.  -4. 676,  -3.190,  8.363,  O.B61,  -4.338)  . 


I ). 


Then  the  best  er-Liirate  of 


K  = 


"  Z(l)>  ^a2  “  *(l),4a  “  ^(1  )>  -*J-  Ca  a2ax  *  *(l)| 


is  [see  Equation  5] 

£»  (a'a)“Vz 


(8) 


and 

£'  “  [5-356,  0.678,  -1.621,  5.488,  6.523,  4.880,  4.91b,  -0.646, 
-I.130,  8.088,  3.l8l,  5.232,  4.198,  4.781,  4.085]  . 

The  analysis  of  variance  corresponding  to  that  of  Table  IV  for  factorial  end 
order  effects  is  given  in  Table  X. 


SV 


Factors 


Orders 


Error 


TABLE  X 

Analysis  of  Variance  for  Factorial 

■a 

and  Order  ©Moots  from  2  FSD 


DF _ SS _ 

7  z(A1^AiAi)*lAiiz'  " 

8  zjMAUrV  -  A^(A^A^)- Jz*  -  11.001 

3  z(l  -  aCa'a)**1*'^'  -  6.193 


Total 


18 


z'z  -  158.394 
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The  eest  of  significance  for  order  effects , 


11. 001) 
8(6.193) 


0.666  , 


indicates  that  the  effect  of  order  is  negligible. 


Since  the  order  effect  is  concluded  to  bo  negligible,  the  beet  estimate  of 


T) 


\\  '  "(D'  \  -  "(1)'  \  -  ’ll)' Va.,  -  ’ll). 


\«3  '  rV  )'  \«3  '  ’ll)'  \a2a3  *  ’ll)) 


IS 


Tj  *  (A^A,)"1^  , 


and 


A  l 
M 


} 5.160,  0.584,  1.331,  3-380,  5.225,  -0.694, .4.927 


J 


The  estimates  of  the  factor  effects  are  then  provided  by 


or 
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\  ' 

l-l-l  l  1-1 

i" 

"3.531*" 

*». 

•1  l-l  1-1  1 

1 

0.883 

\ 

-l-l  1-1  1  1 

1 

-0.T48 

V. 

•  £ 

-l-lll  -1  -1 

1 

O 

m 

•0.386 

*y  y 

’lY3 

-1  1-1-1  1  -1 

1 

0.538 

1  -l  -1  -1  -1  1 

1 

-0.117 

*Wi 

t  ■ 

111-1  »1  >1 

0 

1 

m 

-O.lM* 

1  ■ 

vtrlonoo  of  i  1*  lltlMMi  by 

&  (I)  • 


'0.636  -0.189  >0.189 

O.636  -0.189 

0.638 


0  0 

0  0 

0  0 

o.Vrv  -o.ui 

0.1*79 


0  -0.086*1 

0  -0.006 

0  -0.006 

-O.UI  0 

-O.UI  0 

0.1*79  0 
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Tecta  of  significance  for  the  feet or*  and  Interaction*  ere 


VXSt(3) 

■$&&’  6'939 

V2!t(3) 

-  -feUL  -  0.35? 
Vo.  636 

V3!t(3) 

.  &M-.  .  -0.938 

VOT236 

ViV*(3> 

-  m  -0.4T1 

V0A79 

V3rt(S) 

-  m  0.803 

VO.VfS 

V3st(3) 

-  a  -0.l6<; 

V  0.1*79 

W3“(3) 

-  -  -0.220 

Vo.  430 

If  the  0.0';  lev  cl  of  signlfiosnoe  were  used,  the  only  significant  effect  vould 
be  tho  main  effect  V^. 

Tt  in  enl.i entering  to  coapare  the  usual  2^  factorial  design  8r  unite, 

with  on  Fiji,  6r  units,  where,  for  each  design,  a  unit  la  tested  four  times. 

The  variance  of  the  main  and  Interaction  effect*  far  on  PD  ea  described  la 


vox  (rD)  -  ^  +  i^2  j 


420 


Deulgn  o i  Experiment* 


The  voriwiee  of  n*i"  #ff#et*  for  *n  ISD  i*  At^32fi  a£.  Hence,  the  relative 
efficiency  of  th#  I6D  to  th#  ID  for  n*in  effect*  ii 

a2' 

»f  ■v£"tBfa- 0,405410 1*k$ • 

Honoe,  if  o*#  >0.37,  th*  IS D  1*  mom  #ffiol#nt.  Th#  v*rt#ao#*  of  th#  inter¬ 
action  effect*  for  #n  ISD  amt  lou  thin  th#  v#ri*ao#  of  th#  M#in  #ff#ati.  Thu*, 
tho  advantage  of  th#  7BD  1#  #v*n  more  ja»nouao#d  for  th#  iat#r#ctioa  #ff#ot*. 


ESTIMATION  OF  ERROR  SPEOTRA  FROM  THE 
CROSS-AUTOCOVARIANCE  FUNCTIONS  OF  DIFFERENCES 


D.  B.  Duncan 
John*  Hopkins  University 

and 


W.  T.  Walls 

Pan  American  World  Airways 
Guided  Missiles  Range  D. vision 
Patrick  Air  Force  Base,  Florida 


I.  INTRODUCTION  .  The  problem  here  Is  that  of  estimating  the  error 
(noise)  spectra  of  several  tracking  systems  all  of  which  are  simultaneously 
observing  the  same  trajectory  (signal) .  It  extends  the  work  of  a  previous  memo¬ 
randum  (Duncan  and  Carroll,  Cfl  )  from  processes  of  one  dimension  to  processes 
of  several  dimensions.  The  basic  principle  la  similar  In  that  the  estimates 
are  obtained  under  the  requirement  that  they  be  Independent  of  the  trajectory, 

A  slight  change  In  approach  Is  made;  the  estimates  are  obtained  through  dif¬ 
ferences  between  the  processes  rathor  than  through  their  average  processes. 
Either  way  leads  to  a  mathematically  Identical  estimate;  the  change  has  bean 
made  ns  a  matter  of  convenience, 

The  basic  Ideas  can  be  formulated  In  a  relatively  Dimple  oontex*'  as 
follows.  Lot 


y|  ■  m  +  ej,  y2  ■  m  »■  e2,  y3  *  m  +  o3 


be  three  unoorrelated  observations,  all  with  the  same  expected  mean  m  ana 
with  errors  having  the  variances 


Vj  -  vtej),  -  vlej),  v3  -  v(e3). 


In  this  simple  context  the  problem  Is  that  of  estimating  vj,  v2,  v3  without 
knowledge  of  m  . 
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Form  the  difference* 


d12  -  yj  -  y2  -  oi  *e3  ,  dl3  “  Vi  -  V3  “  “  *3 

and  then  the  product 

(1.1)  Pi  »  djjdjj  -  e*  -  Vj-e^a  +  ejeg. 

Since  the  error*  ere  uncorrelated  it  Lb  immediately  apparent  that  pj  Is  an  un¬ 
biased  estimator  of  v.  and  1*  not  dapendent  In  any  way  on  the  oommon  ax- 
peoted  mean  m  •  Similarly 


P2  ■  <12.^23  and  P3  *  t*3 1^3 2 

where  djj  -  dt  -  dj  provide  corresponding  unbiased  eatlmate*  of  v3  and  v^. 

Xhl*  approach,  the  csaeno*  of  which  la  due  to  Grubbs  GO  ,  1*  the  basic 
feuti  re  of  the  method  heie  considered  for  estimating  tha  noLae  apaotra  of  thrae 
or  more  ayatama  Independently  of  the  underlying  common  signal. 

rite  variance*  or  standard  errors  of  the  spectral  eatlmate*  obtained 
Indirect1/  through  the  differences  are  neturelly  not  as  low  at  those  of  th* 
estimates  which  would  be  obtained  directly  from  the  error  processes  them¬ 
selves  if  the  latter  were  known  free  of  the  trajectory.  The  variances  of  tha 
Indirect  estimated  can  be  obtained  by  a  technique  which  In  the  simple  ooniexl 
work  e  u  'allows . 

Since  the  errors  themselves  are  uncorrelated,  the  terms  in  the  right 
hand  side  of  (1.1)  are  all  uncorrolatsd  and  hanoe 


v(pj)  «  v(e£ )  +  v(eie2)  +  v(*ie3)  +  v(o2e3) . 


Woiklng  further  with  each  term 
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v(ef )  “  mM  -  vf  , 
v^ej)  -  v^Vj  ,  each  l,  1, 

and  thus 

vlpj)  »  m14  -  vj  +  vjv2  +  vtv3  v2  v3  . 

If  tho  errors  are  Gaussian.  the  fourth  moment  m14  la  3v2  and  v(pp  and 
the  other  corresponding  variances  become 


v(pp  f  2v2  +  v^v2  +  +  v2v3 

(1.2)  v(p2)  "  2v^  +  vj  v2  +  vtv3  +  v2v3 

v(p4)  «  2v3  ♦  vt  v2  +  v^Vjj  +  v2v3 


II.  THE  PROBLEM.  The  main  problem  will  be  now  stated  in  more  detail. 
Let  t  *  1 .... ,  n 

(yitmmt**ltj  '  f-'it  •  "t  f  °2t]  “nd  [nt  +  j  * 

up  three  processes  of  observations.  Each  has  the  same  underlying  process  of 
o;;;>eoted  means  /mt|  .  The  error  proooosoo  /olt^  ,  /e2tj  and  ia»J  ha>f" 
means  se.ro  (as  their  name  implies),  are  stationary  with epeotrai  deiTsltlce 
ilA(f) ,  S2(f)  and  S3(f),  and  are  unnorrelated  one  with  another.  Each  process  to 
a  vector  process  with  W  components.  That  ic 
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eta. ,  which  without  loss  of  generality ,  wa  will  often  dlaauas  with  k  •  2  or 
k  -  3  .  For  k>  1  the  ipeotrel  densttlea  at  each  frequency  form  kxk  complex 
metrical  which  we  shell  write  aa 


(3.1)  St(f)  -  Rt(f)  +  lQjtf), 


l  *  1,  2,  3 


or.  In  more  detail  for  say  a  two-dimensional  process  with 


Stlltf) 

Si2i(f)  8122^) 


Rul(f>  R112(f> 
Ri2i(f)  R^2^ 


+1 


9lU« 

Ql21(f> 


The  1  ooefflolent  for  the  Imaginary  matrix  Q(f)  la  not  to  be  oonfuaed  with 
the  aubaorlpt  1  for  the  Ith  aystam. 


The  problem  La  to  get  unblaaad  oatlmat.es 


(3.2)  ?!tf )  -  ?j(f)  +  l^(f)( 


l  *  ).,  2,  3 


for  tM  densities  (3.1)  which  depend  In  no  way  on  the  mean  process  £ m.J  . 

A  seci...idary  problem  Is  that  of  estimating  the  variances  or  standard  errors  ol 
Hie  elements  of  the  estimates  and  ^(f),  l  «  1,  2,  3. 

III.  THE  METHOD  .  The  method  for  the  main  problem  Is  first  to  get 
the  dtuerence  processes 


d12t  “  ylt  *  y2f  dl3t  “  Vit  -  y2t<  t-l(...(n 


end  then  to  compute  the  croes-autooovartanoes, 
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Cllh  "  fi^TT 

l 

t-1 

d12td'l3(t+h)  ' 

Cl2h“  nhi 

n-h 

z 

t-1 

d13t^'i2(t+h)  *  h-0,.. 

m 

each  of  which  Uskxk  matrix. 

Than  following  a  k- variate  generallaation  of  the  Bleokman-Tukey  Q3 
spectral  density  estimation  method  (with  Hanning  smoothing)  the  autgoovarlances 
are  next  transformed  to  raw  spectral  density  estimates 


?t0)  -  *i(i)  +  t^O)  . 


where 


*{J)-aAt  [ono+  mfl  (Ciuh  ♦  Ol2h)oo»  “m*  ♦  oo«trrj)l  . 

h»l  * 


'm-1 

l  Wllh  "  °12h)Dln  °#- 

h«l 


3(J)  •  2 At. 

The  real  parts  are  then  rrnocihed  to  the  final  form 


J  •  P, . . .  i  m 


#1  (o)  -  ^  fy(o)  +  ^  R^U) 

KjO)  RjO+1).  J-l . . 

Kl(m)  -  ^Wj(m-l)  +  ^^i(m)  . 

The  imaginary  parts  are  smoothed  in  the  same  way.  The  combined  forms 

sA 0)  -  Kx0)  ♦  i^U) 
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at  J  ■  0, . , , ,  m  cycles  par  record  length  of  n  point*  estlmata  tha  apaotral 
densities 


S2  (f)  •  R|(f)  ♦  IQ^f) 

at  f  ■  Atj/2m  ,  cycle*  per  second,  whara  At  ts  the  time  interval  between 
data  points. 

Similar  aatimata*  are  next  obtained  for  S2(f)  and  83(f)  .  Tha  starting 
difference  processes  using  the  same  notation  are  djit  .  d23t,  and  d31t,  d32t! 
the  cross  autcoovarlances  are  C22h,  llnc*  °3ih'  ^32h*  h  *  0, . . . ,  rh 
and  the  final  netlmataa  are 


^(f)  ■  ff2(f)  +  iSfett)  and  Islf)  -  R3(f)  +  l?S3(f) 


respectively. 


IV.  AN  EXAMPLE  .  Three  independent  stationary  Ctausslan  error  3x1 
vector  prooesnue  [nlt  ^  ,  (*2tJ  •  {*3tj  >  o£  n"2,0i'>Q  time  points  each 
were  generated  using  the  recursive  formula 


"t  “  Avyt  “  BiVi 
3x3  3x3 


'i*t-2  '  B3°t-3  • 
?::3  3x3 


The  processes  wt  from  which  these  were  derived  consisted  of  independent 
tiaueslan  standard  white  noise  3x1  vectors.  The  values  of  the  coefficient 
matrices  for  each  process  were  as  shown  in  Table  .!. 
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TABLE  1 

Coefficients  for  Generating 
Error  Processes 


Proaes* 

(1) 

Proonss 

(2) 

Procass 
|  (3) 

1  0  0 

1 

0  .  0 

1 

0  0 

A 

0  10 

0 

1.1  ,2 

0 

1.1  .2 

0  0  1 

0 

0  1.2 

0 

0  1.2 

-.6 

0  0 

-.4 

0  0 

*1 

0 

0 

-.7  •  .1 

0 

-.45  -.OS 

0 

0  -.5 

0 

0  - .  35 

-.4 

n  o 

b2 

0 

0 

0 

-.45  -.05 

0 

0  -  .  ■>  6 

.16 

0  0 

0 

0 

0 

.  1  /  .  I1 7 

0 

0  .15 

Error  process  1  Is  itself  simply  standard  white  noise.  The  otho.  two 
tie  autoregressive  processes  of  order  1  and  3  respectively .  Tu  get  things 
started  In  the  latter  two  cases,  values  of  et  for  t£0  were  filled  with  white 
noise.  To  eliminate  transient  effects  as  much  as  seemed  desirable,  40  vector 
observations  ware  discarded  from  the  beginning  of  the  processes.  The  comput¬ 
ing  work  for  generating  the  white  noise,  the  error  processes,  <md  subsequent 
calculations  will  be  written  up  In  reports  by  D.  B.  Duncan,  E.  E.  MoGahee 
and  S.  B.  Burkett,  and  by  I..  B.  Collins  and  A.  Rinaldi  who  hava  bean  cooperating 
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In  the  programming  work.  (As  currently  planned,  these  reports  will  appear 
as  Pan  American  Technical  Staff  Memos  and  RCA  Data  Reduotlon  Programming 
Memos  respectively .) 


Threo  sets  of  spectral  density  estimates  Sj(f),  S2(f)  and  83(f)  ware 
obtained  by  the  cross- spectrumof- differences  method  described  in  Snotlon  lit. 
(In  the  example  no  mean  process  was  added  to  eaoh  of  the  error  processes 
since  the  differences  could  be  and  were  obtained  Just  as  well  directly  from  the 
errors,  a.  g.  j d12t  ■  y  »  V2t  "  *lt  "  a2tJ  *  )  maximum  lag  taken  In  the 

example  Wei  m  *  20;  the  time  interval  assumed  was  At  -  .05;  estimates  of 


density  were  thus  obtained  at  f«0,  .5,..,,  10  cycle  a  per  second.  The 
results  are  shown  In  nine  tables  for  eaoh  of  the  processes  lejJ  ,  \ e2t?  end 

io»a  .  Tables  1.1,...,  1.9  show  the  results  for  the  spectral  densltltas  of 
procVss  one,  Table  1.1  la  for  the  1,  1  element,  Table  1.2  for  the  1,2  element, 


and  so  on  up  to  Table  1.9  for  the  3,3  element.  The  columns  marked  (5),  In¬ 
direct  Estimates,  8  2  ,  show  the  real  part  of  tho  estimate  R2(t)  under  the 
Fortran  notation  SR2  and  the  Imaginary  part  $2(f)  under  812,  for  each  of 
the  frequency  f,  denoted  In  Fortran  under  F,  Column  (1).  Tables  2,1,...,  2.9 
and  3.1,.,.,  3 .9  In  Column  (5)  show  the  corresponding  results  for  processes 


two  and  three , 


For  comparison,  the  columns  hoaded  (2),  Expected  Values,  8,  show 
the  1  sal,  8 R,  and  Imaginary  part,  31,  of  the  spectral  density,  8(f),  being 
estimated;  ths  columns  headed  (3),  Dtraot  Estimates,  31,  show  the  real,  Ski, 
end  itrmolnary  part,  311,  of  the  estimates  obtained  directly  from  the  Individual 
proao-ms,  which  are  available  in  «  simulation  study  Ilka  this,  but  which,  of 
course,  would  uul  uu  available  In  n  real  problem.  The  direct  estimates  represent 
an  optimum  which  one  could  not  hop-  to  lmpiove  upon.  The  standard  errors  for 
the  roul  and  Imaginary  parts  cf  both  the  direct  estimates  and  tho  Indtr-ot  estimates 
have  been  computed  by  the  formulae  developed  in  the  subsequent  sections  end 
are  shown  in  columns  (4)  and  (6)  under  the  Fortran  notations  E1U,  Ell  ami  ER2, 

EI2  ru&pmjllvely , 

V  EXPECTATIONS  AND  VARIANCES  OF  ESTIMATES.  In  this  Station  w- 
derlve  the  expectations  (showing  '.hat  they  are  unbiased)  and  the  approximate 
variances  ol  tho  spectral  density  estimates  obtained  through  the  difference 
processes.  In  so  doing,  to  help  introduce  the  methods,  we  also  first  darlva 
the  well-known  expeotations  and  approximate  variances  of  the  direct  estimates. 

In  each  part  the  work  Is  considered  In  detail  for  the  case  r  -  1,  where  r  le 
the  ratio  r  -  n/m,  In  which  car, 9  a  spectral  density  estimate  oan  be  regarded 
approximately  an  the  squared  modulus  of  a  single  complex  random  variable. 


* 
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The  results  then  oarry  over  to  the  case  r>  1  by  virtue  of  the  faot  (Jenkins  [+]  ) 
that  an  estimate  in  this  case  can  be  expressed  approximately  as  the  mean  of  r 
unoorrelated  estimates  of  the  case  r  -  1  form.  Thua  the  expectation  U  the 
same  and  the  variance  la  obtained  simply  by  dividing  by  r. 

s.i  BlBSfiX  ESTIMATES.  UNIVARIATE  PROCESS.  Aim:  To  shew  that 
E  [s(ff]  -  8(f).  v  8(f)  -  J  S2(f) 

where  s(f)  represents  a  direct,  aay,  Blackman  and  Xukey[£l  estimate  of  any 
one  of  the  error  spaotral  denaity  functions . 


Case  r“H  Hero  we  can  write 

'S(f)  -  za 
app^x. 

where 

*  "  <x  +  ly*’  “  °‘  VM  "  v*y)  *  s(f)' 

and  c(x,  y)  •  0,  (using  c(  •  ,  ■)  for  o(‘  ,  •> ). 
r 

t 

jNott  In  parsing:  This  impM.es 

Sm 

E(z)  -  ^.fitx)  +  1  E(y)  -  0, 

V(z)  -  E(z  -  E(x))  (z  -  E(z»,  by  definition 


Thus, 

E 


E(zS)  ■  -j  E(x2  +  y^) 
j  (8(f)  +  8(f))  -  3(f).  J 


8(f) 


*  EUl)  *  v(jd)  "  S(I) 
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whtoh  established  the  first  result.  Next, 
v  $f)  -  v^(x2+yz) 

If  x  and  y  (and  thus  z  )  are  Gaussian,  which  ws  shall  also  assume,  we  have 
v(x2)  «  v(y2)  «  2Hz(f), 

Also  x,  y  aro  independent  from  which  o(x  ,  .  0.,.  Hepoe 

(2  S2U)  +  2  32(f))  -  S2(f) 

which  establishes  the  second  and  final  result. 

Qarq  r  >  li  Here  the  estpeotatlon  is  the  sama  as  for  ossa  r  ■  1  ,  the 
varunnw  is  obtained  approximately  by  dividing  by  r  «  n/n.  ,  thus 

’[H  -  8(f),  '/  -  J-82(f). 

A  &. 2  DIRECT  EaiiiVlivl 6b.  K- VARIATE  PROCESS  ■  Aim:  Put 
«!,<()  “  fttj(f)  +  iOy(f)  for  the  d<reot  estimate  of  the  ij"1  cross- upeotrel 
density  S^(f)  “  R^f)  +  IQij(f)  for  any  one  of  the  error  prooeasaii.  Than 
It  is  d>'  sired  to  show  that 

B  Rlj(f) 

and 

v[  *„#) 

V  ["  0|j(f) 


-  Rtj(f).  e  $tJ  <fl  •  QyW 

-  -^“(a  +  b) 

■  (A  <-  B)  . 
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A  -  Ru(f)  Rjj(f)  and  B  *  R^(f)  -  Q*(f) , 

[Note;  Strictly  speaking,  the  development  here  for  r  ■  1,  la  not  oorrect  for 
the  extreme  frequencies  f  ■  0  and  f  *  m/2  ,  At  each  of  these  frequencies 
the  spectral  variable  z  is  real  and  may  be  written  as  *  “  x. 

From  this 


Z 


v(z)  ■  S(f) 


and  S(f)  Is  unbiased  as  before,  However,  for  the  variance 


which  is  double  the  previous  velus  S2(f)  .  The  standard  error  la  4TS(f)  , 

Because  of  the  smoothing  used  In  the  more  general  type  of  spectral 
estimate  for  the  cases  r>  1,  the  standard  errors  (l/<Tr)S(f)  for  f  -  0  and 
*r/2  ere  not  low  by  nearly  as  much  as  the  factor  l/JT  and  this  formula  has 
b»*n  used  for  all  frequencies. 

Remarks  of  this  type  apply  to  tha  derivations  of  all  standard  errors  In 
th ) s  report  at  the  extreme  frequencies  f  ■  0  and  m/2  ,  In  looking  at  Tables 
l.i,...,  3.9  the  standard  errors  shown  for  both  the  direct  and  Indirect  uutJ'nate 
at  f  -  0  and  m/2  are  on  the  low  side  for  this  reason.] 

Case  r  ■  h  Hare  we  can  wrlta 


(f) 


■l1! 


whero 
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z*m4 rK  +  1  y k>,  B(x^)  -  0,  v(xk)  -  v(y)t)  -  Rkk(f), 
o(xjt,  yk)  -0,  k  ••  1,  j!  o(xt,  Xj)  -  o(yt,  Vj)  *■  R^f), 
-o(xl(  yj)  «  c(Xj,  yt)  »  QtJ(f). 

Thus 

fljjtt)  -  Re(^j(f))  ■  +  yiVj)  . 

Qtj(t)  *  Im(ty(f))  (-x^  +  Xjyt)  . 

Now 

-  |(E(xlxj)  +  E(ylyj))-  ^(2RlJ(f))s*RlJ(f) 

-  ^  (~E(xiyj)  +  Etx^^J  -  2(2Qlj(0>  "  QU(£) 

vfhloh  establishes  the  result  on  unbiasedness. 

Nf-x.. , 


M 


V 


•^{v(xtXj) 


4-  2o(XjXj, 


y^j)  +  vty^j)), 


To  expand  this  we  will  use  a  result  due  l.u  IsswiUss  quoted,  tor  example, 
by  ftlacikmen  end  Tukey  fl,  p.  100  1  i  If  a,  b,  a,  d  ore  tour  Gaussian  vari¬ 
ables  with  aero  moans,  ttien  J 

o(ab,  od)  ■  E(aoj  E(bd)  4-  E(ad)  E(bo) , 


First, 
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vtx^p  -  o(xiXj .  XjXj) 

-  Ru(f)  Rjj(f)  +  kfjW, 

)  »  0  -  Qy  (f) , 

v{ylyj'  +  Rl2J(f)> 

Combining,  we  get 

v(fitJ(f))  -  -J(2  Ru(f)  Rjj  (f)  f  2R*J(f)  -  2  qj(f))  -  \  (A  +  B), 


which  establishes  the  first  result  on  variances. 
Next, 


«l,W 


|(v(*iyj) -  aoCxiy,.  x.y,)  *  vtxjy,)) 


In  which 


v(x1yJ)  -  Ru(f)  RJ(U)  +  Qy(f)  -  v<Xjy1), 
c(«ty  .  ^yt)  -  Rt2,(0  +  0, 

■ji-'lng 

v  [  Qu{f)]  (A  -  B) . 

which  concludes  case  r  “  1. 


Case*  r>  1;  Here  the  results  arc  the  »*me  as  for  r  »  1  except  for 
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the  additional  1/r  factor  In  the  variances,  establishing  the  desired  results. 
Diagonal  terms;  For  these  It  Ib  noted  that 


v[  Vf)]  -  ir(2Ru(f)  -  ?Ru< 

V  *  o 


which  agree,  os  they  should,  with  the  univariate  obbo  results. 


Atm:.  Putting 


SljkW  "  Ruic(f)  4  ^ik^ 


for  the  Indirect  estimates  defined  In  section  3,  for  the  j  kth  speotral  estimate 
at  frequency  (f)  for  system  1;  1-1,2,  3;,),  k  **  1, . . . ,  K,  It  Is  desired  to 
show  that 

E  [  V>]  '  V11' 

“•••  E  -  V'1'  E  [«uk“l  -  V«>, 


end  that 


v  f  Vf)l  "  Vr(A+B  +  C) 

v  I^JkW)  -  2f  (A  -  B  +  C) 


A  *  Rtj]  Rlkk'  E  ‘  Rf)k  "  °?)k 
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135 


°  “  Rljj  R3kk  +  r2JJ  Rlkk  +  R2J j  R3kk  ' 
for  1  ■  2, 

C  “  R2JJ  R3kk  +  RljjR2kk  +  R1JJ  R3kk  ' 
and  for  1  ■  3, 

c“R3jjR2kk  +  Rl]jR3kk  +  Rl)jR2kk  1 

Case  r  ■  li  Here  we  oan  write  for  system  1  [  1.  a,  1  ■  l]  , 

Sljk<f>  “a12j513k  ”  <al)  -  a2i»ik 

“  *1.1  alk"  alj*3k  •  a2J*lk  +  *2j*3k 

where 

zhj  “iKj  +  lvhk)-  zu  ■  + 

and  the  real  x's  and  y's  have  the  same  means,  Variances  and  covariances 
within  each  system  (l .  a. ,  fnr  h  -  1)  as  given  In  the  previous  subsevtlon . 

In  addition  all  covariances  across  systems  (l.  e. ,  for  h  f  i)  are  zero. 

First,  dealing  with  real  and  Imaginary  parts  together, 

E(S1)k(f))  -  E(z,j  zlk)  -  0  0  +  0  =  3ljk 
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which  establishes  the  result  on  unbiasedness.  Next, 


v(S1)j4{f))  -  +  v(x4jl3k)  +  v(x2jClk)  +  v(z2jz3k), 

all  cross  product  terms  being  zero  because  of  the  zero  crossproducts  between 
systems.  Dealing  with  Just  the  real  parts  of  the  right  hand  side,  term  by  term, 
we  have 

vjRelz^S^);)  "WlR^)  (of.  previous  subsection) 

-  f  A+-JB,  (1  -  1), 

v(Re(zjj  z^))  -  v(|(XjjX3k  +  yijy3k>) 

"  4<R1JJ  R3kk  +  2(0  +  05  +  R1JJ  R3kk> 

"  4R1J)  Rikk  ' 

SI  mi  i.irly 

v(Re(z2j-rik)  -  T  RZJJ  Rikk 

v(ReU2j2'3k))  -  2"  r2]J  R3kk  1 
Collecting  terms 


v(Rljk)  -  -jA  +  ^8  +  -|c, 


U  *■  1). 
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This  establishes  the  raault  for  v(R, J)t).  Similar  steps  lead  to  the  corresponding 
results  for  1-2,3. 

Finally,  for  the  imaginary  terms 


vUmUjj  z1)t))  -  v(^ij)c) 


-  ■g  A  -  -j  B , 


(of.  previous  subsection) 


(1-1), 


v(Irn(*ij  z^))  -  v(  ^  (-x1jy3|t  +  y1Jx31c)) 


■  4  (R1)J  R3kk  +  2^°  +  +  R1JJ  R3kk) 

*  iRlJJ  R3kk  '  0t0, 


Collecting  terms 


(l  -  i)  . 


This,  and  similar  steps  for  1-2,3  establish  tho  (r  -  1)  -  case  result 
nn  variances. 

Case  r>  1;  The  (r  *=  1)  -  oaoe  result  on  expectations  tog*i'.‘..«r  with 
the  variance  results  divided  by  r,  establish  the  full  (r  ^  1)  -  case  results. 

5.4  RTTMMARY  OF  RESULTS  ON  VARIANCE3  .  The  results  of  this  sec¬ 
tion  which  have  been  used  in  computing  the  standard  errors  (square  room  of 
the  variances)  in  Tables  l.l . 1.9  may  be  summarized  as  follow-: 

For  direct  estimates 


v(Rijk(f))  -  (A  +  B) 
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whore 


A.RlJj(f)Rlkk(f),  B-R?JkW-Qgk« 
l  ■  2,  2,  «|  J,  k  -  1,  . . . ,  K 

For  a  diagonal  term  ()  ■  k)  or  a  univariate  (K  -  1)  process  these 
appear  as 

^(f) 

VtdyjW)  «  0 

the  latter  being  reasonable  since  *  0,  l  ■  1,  2,  3 
For  the.  Indirect  estimates 

v(Kijk{f»  -  (A  +  8  +  C) 

vtOijicU/i  -  Jf*-  B*C) 

where  A  and  B  are  as  already  defined  and  for  l  »  l, 


0  ‘  Rljj  R3kk  *  R2jj  Rlkk  +  r2JJ  R3kk  > 


for  l  ■  2 


C  “  R2jJ  R3kk  +  R1J)  R2kk  Rl)j  R3kk' 
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and  tor  1  ■  3 


C  *  R3jJ  R2k)t  +  R1JJ  R3kk  +  Rijj  R2kk' 
1  -  1,  2,  3;  J,  k  -  1, . K. 


For  a  diagonal  term  (j  -  k)  or  a  univariate  K  -  1  process  these  appear  as 


v<Rijj(f»  -  ■“■(Rftjtf)  ♦  2©) 


where 


0  "  Rljj  R2Jj  *  Rljj  R3,1J  +  R2jJ  R3jJ 1 

VI.  PISgVSSIQN  ANP  CPNgIrUPIN9  REMARKS-  The  numerics'  results 
in  Tables  1.1, . . .,  3,9  s'srvu  to  Illustrate  and  emphasize  the  properties  of  un¬ 
biasedness  and  the  theoretical  variances  for  the  Indirect  estimator  obtained  In 
section  V.  These  bear  out  the  conclusions  of  the  previous  jludy  [2]  In  the 
unlvarleto  ease  Y.  -*  1  ,  shoving  that  they  "iso  apply  In  the  more  general  cases 
K>  1. 


The  Indirect  estimates  S?  (or  S2  an  they  appear  In  the  tahlns)  are 
good  (havn  a  relatively  low  standard  error)  el  most  points  and  especially  ul 
frequencies  at  which  powers  In  the  other  systemo  are  lower  than  the  one  being 
estimated,  They  are  not  so  good  however  at  fr«quencies  at  which  the  reverse 
is  truet  At  frequencies  at  which  the  powers  in  the  other  systems  uvu  much 
larger  than  the  one  being  estimated.  The  noise  added  to  the  spectral  estimate 
from  the  other  systems  by  having  to  work  through  the  differences  Involving 
the  latter  can  be  seriously  latya  at  such  points. 

Work  Is  proceeding  on  an  iterative  approach  to  minimize  this  difficulty. 
This  can  be  briefly  indicated  in  principle  as  follows .  After  uue  step  consisting 
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of  the  above  procedure  and  on  the  basis  of  the  knowledge  so  gained  about  the 
error  speotrum  In  each  case,  each  system  Is  filtered  to  give  Individually  opti¬ 
mum  Independent  estimates  of  the  trajectories 


t 


") 

n  'j  ,  1  *  1 ,  2 ,  3 . 


The  Indirect  speotrum  estimation  procedure  can  then  be  repeated  In  a  seoond 
step  based  on  the  differences 


l12t  “  Ylt  ■  Y2t‘  ^lat"  Ylt  "  ^3t:  t-1'" 

, ,  n  |  for  system  1 

-V 

‘21t  “  ^2t  "  Ylt‘  d23t "  Y2t  “  ^3t!  t“1'" 

. ,  n  t  for  system  2 

l 

‘3  It  ■  V3t  “  Ylt*  d32t“  Y3t  "  Y2t* 

, ,  nt  for  system  3 

If  the  new  upeotral  estimates  3how  muoh  oho  ige,  the  same  step  can 
be  repented  again  and  several  times  thereafter  If  necessary. 

The  same  method  expressed  In  termB  of  residuals  (as  in  QQ  )  extends 
readily  to  coses  with  more  than  three  systems  .  The  greater  tha  number  of 
systems  the  less  the  cross  tratiSiYsl-elon  of  noise  to  the  Indirect  rn-jutral  estimate. 


To  sum  up,  the  given  method  will .  In  itself,  provide  a  good  speotrel 
estln.au->  In  many  situations,  in  most  of  the  remainder  It  promise  .  to  ^.o'drie 
a  valuable  first  stop  In  Iterations  leading  to  a  good  estimator. 
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REALISTIC  EVALUATION  OF  THE  PRECISION  AND  ACCURACY 
OF  INSTRUMENT  CALIBRATION  SYSTEMS* 


Churchill  ElPflnhart 

National  Bureau  of  Standard*,  Washington,  D.  C, 


ABSTRACT.  Calibration  of  Instruments  and  standards  la  a  refined 
form  of  measurement,  Measurement  of  some  property  of  a  thing  U  an  oper¬ 
ation  that  yields  as  an  end  result  a  number  that  indicates  how  much  of  the 
property  the  thing  has,  Measurement  Is  orrilnadly  a  repeatable  operation,  »o 
that  it  is  appropriate  to  regard  meansurement  as  a  production  proaess,  the 
"product"  being  the  numbers,  l.e  the  measurements,  that  it  yields;  and  to 
apply  to  measurement  processes  In  the  laboratory  the  oonoepts  and  techni¬ 
ques  of  statistical  process  control  that  have  proved  bo  useful  In  the  quality 
control  of  Industrial  production. 

Viewed  thus  It  becomes  evident  that  a  particular  measurement  operation 
cannot  be  regarded  a*  constituting  a  measurement  process  unless  statistical 
stability  of  the  type  known  as  a  state  of  statistical  control  has  been  attained. 

In  order  to  determine  whether  a  particular  measurement  operation  Is,  or  Is  not, 

In  a  state  of  statistical  control  It  Is  necessary  to  be  definite  on  what  variations 
of  procedure,  apparatus,  environmental  oondlttona,  observers,  operators,  etc., 
.ire  allowable  In  "rupooteri  Applications"  of  w'-’ot  will  be  considered  to  be  the 
same  measurement  process  applied  to  the  measurement  of  the  same  quantity 
under  the  same  conditions,  To  be  realistic,  the  "allowable  variations"  musi 
be  of  sufficient  scope  to  bracket  the  clrcumstanoes  likely  to  be  met  In  practice, 
furthermore,  any  experimental  program  that  alms  to  dutermLne  the  standa’-1.  de¬ 
viation  of  a  measurement  p,  press  as  an  Indication  of  Its  nr-idslon,  must  be 
based  on  appropriate  random  sampling  of  this  llhflly  range  of  olroumstunces. 

Ordinarily  the  accuracy  of  a  measurement  process  may  bu  cll*'n>cterl*ed 
by  giving  (a)  the  standard  deviation  of  the  procosn  and  (b)  credible  bound#  L,: 

Us  likely  overall  systematic  error.  Determination  of  credible  bounds  to  llie 
oomblnod  effect  of  recognized  potential  sources  of  systematic  error  always 
involves  some  arbitrariness,  not  only  In  the  placing  ol  reasonable  bounds  on 
'ho  systematic  error  likely  to  be  contributed  by  each  particular  as  "iynable 


^Presented  at  the  1962  Standards  Laboratory  Conference,  National  Bureau  of 
Standards,  Boulder,  Col.,  August  8-10,  1962.  Published  In  the  Journal  of  Re¬ 
search  of  the  National  Bureau  of  Standards--C,  Engineering  and  Instrumenta¬ 
tion,  vol,  67G,  No,  2,  Aprll-June  1963. 
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cause,  but  also  in  the  manner  In  which  these  individual  contributions  are  com¬ 
bined,  Consequently,  the  "Inaccuracy"  of  and  results  of  measurement  oannot 
be  expressed  by  “oonfldenca  limits"  corresponding  to  a  definite  numerical 
"confidence  level,"  except  In  those  rare  Instances  In  which  the  possible  over¬ 
all  systematic  error  of  a  final  result  Is  negligible  ir.  comparleon  with  Its  im¬ 
precision  . 

1,  INTRODUCTION,  Calibration  of  Instruments  and  standards  Is  bas¬ 
ically  a  refined  form  of  measurement.  Measurement  Is  the  assignment  of  nuni- 
buis  to  material  things  to  represent  the  relations  existing  among  them  with  re- 
spool  to  particular  porpartlea,  One  alwaya  measures  properties  of  things,  not 
the  thing*:  themselves.  In  practice,  measurement  of  some  pruperty  of  a  thing 
ordinarily  takes  the  form  of  a  sequence  of  steps  or  operations  that  yields  as 
an  oik  result  a  number  that  Indicates  how  much  of  this  property  the  thing  haa, 
for  so  -one  to  uee  for  a  specific  purpose,  The  end  result  may  be  the  out¬ 
come  ut  a  single  reading  of  an  Instrument,  More  often  It  la  some  kind  of  aver¬ 
age,  e.g, ,  the  arithmetic  mean  of  a  number  of  independent  determinations  of 
the  same  magnitude,  or  the  final  result  of  a  least  squarss  "reduction"  of  measure¬ 
ments  of  a  number  of  different  quantities  that  bear  known  relations  to  aaoh 
other  In  accordance  with  a  definite  experimental  plan  In  general,  the  purpoie 
for  which  the  answer  Is  needed  determines  the  acauracy  required  and  ordinarily 
also  1.1 1 h  method  of  measurement  employed. 

.Specification  of  the  apparatus  and  auxiliary  equipment  to  be  used,  tbs 
operat'd, a  to  be  performed,  the  sequence  In  which  they  are  to  be  executed, 
and  the  ...ondtttons  under  which  they  ere  respectively  to  be  carried  out--these 
instructions  collectively  serun  to  online  a  method  of  measurement ,  A  measure¬ 
ment  procees  Is  the  realization  of  a  method  of  measurement  In  terms  of  parti¬ 
cular  apparatus  and  equipment  of  '.he  prescribed  kinds,  particular  conditions 
that  ot  bust  only  approximate  the  conditions  prescribed,  and  particular  pw.uo’>s 
as  operators  and  observers. 

It  has  long  been  reaoynluud  that,  In  undertaking  to  apply  a  particular 
method  of  measurement,  a  degree  of  consistency  among  repeated  measurement? 
of  a  single  quantity  needs  to  be  attained  before  the  method  of  measure  mu  nt. 
concerned  can  be  regarded  as  meaningfully  realized,  t.e.,  before  a  measure¬ 
ment  process  nan  be  said  to  have  been  established  that  Is  e  realization  of  the 
method  of  mesurement  concerned,  Indeed,  consistency  or  statistical  stability 
of  a  vary  special  kind  Is  required:  to  qualify  as  a  measurement  process  a 
measurement  operation  must  have  attained  what  is  known  In  Industrial  quality 
control  languago  as  a  stai  n,f  atnUstlcel  control.  Until  «  measurement  opera¬ 
tion  has  been  "debugged"  to  the  extent  that  it  has  attained  a  state  of  statistical 
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control  It  cannot  be  regerdod  In  any  logical  sense  as  measuring  anything  at  all, 
And  when  It  has  attained  a  state  of  statistical  control  there  may  still  remain 
the  question  of  whether  it  is  faithful  to  the  method  of  measurement  of  which 
it  is  intended  to  be  a  realization. 

The  systematic  error,  or  bias,  of  a  measurement  process  refers  to  its 
tendeay  to  measure  something  other  than  what  wan  l  .ended;  and  is  determined 
by  the  magnitude  of  the  difference  /a.  -  t  between  thu  process  uveraqe  or  limit¬ 
ing  mean  fx  associated  with  measurement  of  e  particular  quantity  by  the  measure¬ 
ment  process  concerned  and  the  true  velue  r  of  the  magnitude  of  tnls  quantity  , 
On  first  thought,  the  "true  value"  of  the  magnitude  of  a  particular  quantity 
appears  to  be  a  simple  straightforward  concept.  On  careful  analysis,  however, 

It  becomos  ovtdant  that  the  "true  value'1  of  the  magnitude  of  a  quantity  la  In¬ 
timately  linked  to  the  purposes  for  which  knowledge  of  the  magnitude  of  thla 
quantity  is  needed,  and  cannot,  in  tho  tinai  analysis,  be  meaningfully  and 
usefully  defined  in  Isolation  from  these  neudu, 

The  precision  of  a  measurement  process  refers  to,  end  is  determined  by 
the  degree  of  mutual  agreement  characteristic  of  independent  measurements  of 
a  single  quantity  yielded  by  repeated  applications  of  the  process  under  specified 
conditions;  and  tt a  accuracy  reters  to,  and  Is  determined  by,  the  degree  of 
annulment  of  such  measurements  with  the  true  value  of  the  magnitude  of  the 
quantity  concerned.  In  brief  "accuracy"  has  to  do  with  closeness  to  the  truth; 
"precision, "  only  with  olosenoss  together. 

Systematic  error,  precision,  end  aoouraoy  are  inherent  characteristics 
ol  a  measurement  proceas  and  not  of  a  particular  measurement  yielded  by  the 
process.  Wn  may  Mso  upeuk  of  the  systematic  error,  pieotslon,  and  accuracy 
of  a  particular  method  of  measurement  that  has  the  capability  of  statistical 
control,  But  these  terms  arc  not  denned  for  a  measurement  operation  that  is 
not  in  a  state  of  statistical  control, 

The  precision,  or  mure  correctly,  the  imprecision  of  t  measurement 
process  is  ordinarily  summarized  by  the  standard  deviation  of  the  process, 
wnich  exprosses  the  characteristic  disagreement  of  repeated  measurement* 

01  a  single  quantity  by  the  process  concerned,  nnd  thus  serves  to  indloate 
by  how  much  a  particular  measurement  Is  likely  to  differ  from  other  values  that 
the  same  measurement  process  might  have  provided  in  this  instance,  or  might 
yield  on  remeesurement  of  the  same  quantity  on  another  occasion.  Unfortunately, 
there  does  not  exist  any  single  comprehensive  measure  of  the  acouracy  (or 
inaccuracy)  of  a  measurement  procos*  analogous  to  the  standard  deviation  as 
a  measure  of  its  imprecision. 
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To  characterise  the  accuracy  of  a  measurement  process  It  Is  necessary, 
therefore,  to  Indicate  (a)  Its  systematic  error  or  bias,  (b)  Its  precision  (or 
jjapruclsion)--snd,  strictly  speaking,  also,  (c)  the  form  of  the  distribution 
of  the  individual  measurements  about  the  process  average.  Such  Is  the  un¬ 
avoidable!  sMuation  if  one  is  to  cotieorn  one  s  Belf  with  Individual  measure¬ 
ments  yielded  by  any  particular  measurement  process,  Fortunately,  however, 
"final  rosults"  are  ordinarily  some  kind  of  average  or  adjusted  value  derived 
(lorn  a  wot  of  independent  measurements .  and  when  four  oi  moro  Independent 
m.iaiiui  omenta  are  Involved,  such  adjusted  values  tend  to  Ine  normally  dlstrl- 
bnlud  to  a  very  ctnnd  nuuroximatlon .  sn  that  the  accuracy  of  such  final  result* 
can  oidlnarily  bo  characterised  satisfactorily  by  indicating  (a)  llinlr  Imprecision 
on  expressed  by  thalr  standard  error,  and  (bj  the  systematic  error  of  the  process 
by  which  they  worn  obtained 

The  error  of  any  single  measurement  or  adjusted  value  of  e  particular 
iiuantlty  is,  by  definition,  the  dlfferunca  between  the  measurement  or  adjusted 
value  concerned  and  the  true  value  of  the  magnitude  of  this  quantity,  The 
or  rot  of  any  particular  measurement  or  adjusted  value  is,  therefore,  a  fixed 
number)  and  this  number  will  ordinarily  be  unknown  and  unknowable,  because 
the  true  value  of  the  magnitude  of  the  quantity  concerned  is  ordinarily  un¬ 
known  and  unknowable.  Limit*  to  the  error  of  a  single  measurement  or  ad- 
luau'.',  v'.ihie  may,  however,  be  Inferred  from  (a)  the  precision,  and  (b)  bounds 
on  the  systematic  rjrror  of  the  measurement  process  by  which  It  was  produced-- 
but  not  without  risk  of  being  Incorrect,  because,  quite  apart  from  the  tnexect- 
nour  with  which  bounds  are  commonly  placed  on  a  systematic  error  ot  a  mess- 
uiom'Mii  process,  such  limits  are  applicable  to  the  error  of  the  single  measure¬ 
ment  (■•t  adjusted  value,  not  as  a  unique  Individual  outcome,  but  only  os  a 
l/plcal  ofluo  ol  the  crroi.,  cliaiaaterivUa  of  such  meaeureir.Juis  of  the  name 
quantity  that  might  nave  boon,  or  ir'ght  bo,  yielded  by  the  same  measure¬ 
ment  p.vcosa  under  the  oame  conditions. 

dneo  the  precision  of  a  measurement  process  Is  determined  by  the 
ohorjcb.rlBtlo  "closeness  together"  of  successive  Independent  nieunuro- 
moni.M  of  a  single  magnitude  generated  by  ispeated  apollcauon  of  the  process 
under  specified  conditions,  and  Its  blus  or  systematic  error  Is  determined  by  the 
direction  and  amount  by  which  such  measurement#  tend  to  differ  from  the  true 
value  of  the  magnitude  of  the  quantity  concerned,  It  Is  necessary  to  be  clear 
on  what  variations  of  procedure,  apparatus,  environmental  conditions,  ob¬ 
servers  ,  etc. ,  are  allowable  In  "repeated  applications"  or  what  will  be  con¬ 
sidered  to  be  tho  same  measurement  process  applied  to  the  measurement  of  the 
name  quantity  under  the  same  conditions  If  whatever  measures  of  the  precision 
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and  bias  ot  a  measurement  process  wo  may  adopt  ere  to  provide  a  realistic 
tndatlon  of  the  accuracy  of  this  process  in  pmotlce,  then  the  "allowable  varia¬ 
tions"  must  be  of  sufficient  scope  to  bracket  the  range  of  circumstances  com- 
mohly  met  In  practice,  Furthermore,  any  experimental  program  that  alms  to 
determine  the  precision,  and  thence  the  accuracy  of  a  measurement  process, 
must  be  based  on  an  appropriate  random  sampling  of  this  "range  of  alrciumstances, " 
If  the  usual  tools  of  statistical  analysts  are  to  be  strictly  applicable, 

When  adequate  random  sampling  of  the  appropriate  "range  of  circum¬ 
stances”  Is  not  feasible,  nr  even  posslblo,  then  it  is  necessary  (e)  to  compute, 
l>v  extrapolation  from  available  data,  a  more  or  loss  subjective  estimate  of  tho 
preaialon  of  the  measurement  process  concerned,  to  serve  as  a  substitute  for 
a  direct  experimental  measure  of  this  ohereoterlstlc,  and  (b)  to  aealgn  more  or 
less  subjective  bounds  to  the  systematic  error  of  the  measurement  process. 

To  the  extent  that  uuoh  at  least  partially  subjective  computations  are  involved, 
the  resulting  evaluation  of  the  overell  accuracy  of  a  measurement  prooeas 
"It  baaed  on  aubjaot-mattor  knowledge  end  sklU,  general  Information,  and  . 
lntultlon--bi.it  not  on  statistical  methodology"  (Ooohran  at  el,  19 S3 ,  p.  693j, 
Consequently,  In  such  cases  the  statistically  precise  uonoapt  of  e  family  of 
"confidence  Intervale"  associated  with  a  daflntta  "confldanoa  level"  or  "con¬ 
fidence  coefficient"  Is  not  applicable. 

The  foregoing  points  and  certain  other  related  matters  are  discussed 
I,,  greater  detail  In  the  euooeedlng  sections,  together  with  en  indication  of 
pioovriures  for  tho  realistic  evaluation  of  precision  and  accuracy  of  ustuullahed 
procedures  for  the  calibration  of  Instruments  and  standards  that  minimize  as 
.iiu-h  it  possible  the  subjective  elements  of  such  an  evaluation.  To  the  extent 
that  complete  ellmlrtnunn  ot  flu.  subjective  element  Is  rot  always  possible, 
the  responsibility  for  an  Important  and  enmeUmez  the  most  difficult  part  of  the 
evaluation  id  shifted  from  the  shoulders  of  the  statistician  tn  the  shoulders 
ol  the  subject  matter  "expert," 


2,1,  Nature  and  Object ,  Measurement  Is  tho  assignment  of  numhy.s 
to  material  things  to  represent  the  relntlons  existing  among  them  wliii  ruspeot 
to  particular  properties.  The  number  assigned  to  some  particular  property  serves 
to  represent  the  relative  amount  of  this  property  associated  with  the  object 
concerned, 

Measurement  always  pertains  to  properties  of  things,  not  to  the  thine  a 
themselves,  Thus  wu  cannot  measure  a  meter  bar,  but  can  and  usually  do, 
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measure  Its  length;  and  we  could  also  measure  Its  mass,  its  danlty,  and  per¬ 
haps,  also  Its  hardness. 

The  object  of  measurement  Is  twofold:  first,  symbolic  representation 
of  properties  of  thing*  as  a  basis  tor  conceptual  analysis;  and  second,  to  effect 
the  representation  In  a  form  amenable  to  the  powerful  tools  of  mathematical 
analysts.  The  decisive  feature  Is  symbolic  representation  ot  properties,  for 
which  unci  numerals  arc  no!  the  only  usable  symbols, 

In  praotloo  the  assignment  of  a  numerical  magnltudo  to  a  particular 
property  of  a  thing  la  ordinarily  accomplished  by  comparison  wlih  a  set  of 
standards,  or  by  comparison  cither  of  the  quantity  tt.uolf,  or  u!  Home  transform 
of  It,  with  a  provlously  calibrated  settle.  Thus,  length  measurements  ere  ue- 
ually  made  by  directly  comparing  the  length  concerned  with  a  oellbret.od  ber 
or  tape;  and  mass  measurements,  by  directly  comparing  the  weight  of  a  given 
mass  with  the  weight  of  a  set  of  standard  masses,  by  moans  of  a  balance;  hut 
foren  measurements  ere  usually  carried  out  In  terms  of  some  transform,  such  as 
by  reading  on  a  calibrated  scale  the  extension  that  the  force  produces  In  e 
spring,  or  the  deflection  that  It  ptoduuss  In  a  proving  ring;  and  temperature 
measurements  are  usually  performed  In  terms  of  soma  transform,  suoh  as  by 
loading  on  o  calibrated  scale  the  expansion  ot  a  column  of  meroury,  or  the 
clue..; mil  resistance  of  a  platinum  wtiu, 

2.2.  QuoiLlaUvo  and  quunUtjSttv.Q. fttfasiU..  As  Walter  A.  Hhawhart, 
father  of  stattatloal  control  charts,  has  ru  marked: 

"ft  Is  Important  to  realu*> .  ,  .that  there  are  two  aspect*  of  an  operation 
ot  inoasui ament;  one  la  g;;3i.tu«tivo  and  the  other  qualitative,  one  oonalstn 
of  numbers  or  polntoi  readings  such  n«  the  obourvad  lengths  In  q  messuremonts 
of  the  length  of  o  line,  and  the  o»her  cunelsts  of  the  physical  inunLoulntionp 
of  physical  thlnga  by  someone  tri  accord  with  Instructions  that  we  shall  assuiu,.' 
to  be  de>'Crlbable  In  words  constituting  a  text."  ^hewhart  1939,  p;  130, j 

Mote  specifically,  the  qualitative  factors  Involved  In  the  moviursmont  of  a 
quantity  aw:  the  apparatus  and  auxiliary  equipment  (e.a. .  reagents,  batterir,, 
at  other  source  of  electrical  energy,  etc.)  employed;  the  pDerators  and  opsaivaru, 
If  any,  involved;  the  operations  performed,  together  with  the  sequence  In 
which,  r.nd  the  conditions  under  which,  thoy  are  roopeoUvsly  carricid  out. 

2.3,  QfliIflfll.lsiL.aad.Adj3; aimaoLal flbjflpaumi-  The  numbers  obtained 
as  "readings"  on  a  calibrator!  son  In  arc  ordinarily  the  end  product  of  everyday 
measurement  In  the  trades  and  In  tho  homo.  In  scientific  work  there  ate  usually 
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two  important  additional  quantitative  aspects  of  measurement:  (1)  correction 
of  the  readings,  or  their  transforms,  to  compensate  for  known  deviations  from 
ideal  execution  of  the  prescribed  operations,  and  for  non-negliglble  effects  of 
variations  In  uncontrolled  variables:  and  (2)  adjustment  of  "raw"  or  corrected 
measurements  of  particular  quantities  to  obtain  values  of  these  quantities  that 
conform  to  restrictions  upon,  or  Interrelations  among,  the  magnitudes  of  these 
quantities  Imposed  by  the  nature  of  the  problem, 

Thus,  It  may  not  be  practicable  or  economically  feasible  to  taka  read¬ 
ings  at  exactly  ihe  prescribed  temperatures:  but  quite  practicable  and  feasible 
to  bring  and  hold  the  temperature  within  narrow  neighborhoods  of  the  prescribed 
values  and  to  record  the  actual  temperatures  to  which  the  respective  readings 
correspond.  Insuch  cases,  if  the  deviations  from  the  prescribed  temperatures 
are  not  negligblo,  “temperature  corrections"  beeed  on  appropriate  theory  are 
usually  applied  to  the  respective  readings  to  bring  them  to  the  values  that  pre¬ 
sumable  would  have  been  observed  If  the  temperature  In  each  Instance  had  been 
exactly  as  proscribed. 

In  practice,  however,  the  objective  Just  stated  Is  rarely,  If  ever,  actually 
achieved.  Any  "temperature  corrections"  applied  could  be  expected  to  bring 
the  respective  readings  "to  the  values  that  presumably  would  have  been  observed 
If  the  temperature  In  each  Instance  had  been  exactly  as  prescribed"  If  and 
-.,!y  If  these  "temperature  corrections"  made  appro:.: tale  allowances  for  ail 
r.’f  the  effects  of  the  deviations  of  the  actual  temperatures  from  those  prescribed. 
"Temperature  corrections"  ordinarily  correct  only  for  particular  effects  oi  the 
deviations  of  the  actual  temperatures  from  their  prescribed  values;  not  for  all 
of  fhe  effects  on  the  readings  traceable  to  deviations  of  the  actual  r-’mpera- 
tuies  from  those  proscribed.  Thus  Mlchetson  utilised  "temperature  corrections" 

In  his  18/'J  investigation  of  use  speed  of  light;  Dut  his  results  exhibit  a  de¬ 
pendence  on  temperature  alter  "temperature  correction."  The  "temperature 
corrections"  applied  corrected  only  for  the  effects  of  thermal  expansion  due 
to  variations  In  temperature  and  not  also  for  changes  in  the  Index  ot  refraction 
oi  the  air  due  to  changes  in  the  humidity  of  the  air,  which  In  June  and  July 
at  Annapolis  Is  highly  correlated  with  temperature.  Corrections  applied  in 
practice  are  usually  of  more  limited  scope  than  the  names  that  they  am  <r.  »-jn 
a  PE  eu  to  Indicate . 

Adjustment  of  observations  Is  fundamentally  different  from  their  "cor¬ 
rection."  When  two  or  more  related  quantities  are  measured  Individually,  the 
lesulting  measured  values  usually  fall  to  satisfy  the  constraints  on  their  magni¬ 
tudes  Implied  by  the  given  Interrelations  among  the  quantities  concerned.  In 
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such  cases  these  "raw"  measured  values  are  mutually  contradictory,  and 
require  adjustment  In  order  to  be  usable  for  the  purpose  Intended.  Thus, 
measured  values  of  the  three  cyclic  differences  (A  -  B),  (B  -  C),  and  (C- A) 
between  the  lengths  of  three  nominally  equivalent  gage  blocks  are  mutually 
contradictory,  and  strictly  speaking  are  not  usable  as  values  of  these  differences, 
unless  they  sum  to  zero. 

The  primary  goal  of  adjustment  Is  to  derive  from  such  inconsistent 
measurements,  If  possible,  adlusted  values  for  the  quantities  concerned 
that  do  satisfy  the  conc'rutnts  on  their  magnitudes  imposed  by  the  nature  of 
the  quantities  themselves  and  by  the  existing  interrelations  among  them.  A 
second  objective  is  to  select  from  all  possible  sets  of  adjusted  values  the 
bul  lliui  lb  liiu  "beat" — ui ,  at  lecial,  a  set  that  is  "good  enough"  for  the  lntondod 
purpose — In  some  well-defined  sense.  Thus,  In  the  above  case  of  the  measured 
differences  between  the  lengths  of  three  gage  blocks,  an  adjustment  could 
be  effected  by  ignoring  the  measured  value  of  one  of  the  differences  entirely, 
say,  the  difference  (C  -  A),  and  taking  the  negative  of  the  sum  of  the  other 
two  as  Us  adjusted  value, 

Adj(C  -  A)  --[(A  -  B)  +  (B  -  C)|. 

Thl'-  will  certainly  assure  that  the  sum  of  all  three  values,  (A  -  B)  (B  -  C) 

+  Adj  (C  -  A),  Is  zero,  as  required,  and  Is  clearly  equivalent  to  ascribing  all 
of  ihtt  excess  or  deficit  to  the  replaced  measurement,  (C  -  A) .  Alternatively, 
one  might  prefer  to  distribute  the  necessary  total  adjustment  -  |(A  -  B)  + 

(B  -  C)  +  (C  -  All  equally  over  the  Individual  measured  differences,  to  obtain 
the  f o1  lowing  set  of  adjusted  values: 

Adi  (A  -  B)  -  (A  -  B)  -  j  [(A  -  B>  M0  -  C)  +  <C  -  A)] 

-  -j  [ 2 (A  -  b)  -  (D  -  C)  -  (C  -  A)] 

Adj  (B  -  C)  =  i  f  2(B  -  C)  -  (A  -  B)  -  (C  -  A)] 

Adj  1C  -  A)  «  j  [2(C  -  A)  -  (A  -  B)  -  (B  -  C)j  . 

Clearly,  the  sum  of  these  three  adjusted  values  must  always  be  zero,  as  re¬ 
quired,  regardless  of  the  values  of  the  original  individual  measured  differences. 
Furthermore,  most  persons,  I  believe,  would  consider  this  latter  adjustment 
the  better;  and  under  certain  conditions  with  respect  to  the  "law  of  error" 
governing  thn  original  measured  differences,  It  is  Indeed  the  "best," 
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Nota  that  no  adjustment  problem  existed  at  the  stage  when  only  two  of 
these  differences  had  been  measured  whichever  they  wero,  for  then  tho  third 
could  be  obtained  by  subtraction.  As  a  general  principle,  when  no  more  ob¬ 
servations  are  taken  than  arc  sufficient  to  provide  ono  value  of  each  of  the 
unknown  quantities  Involved,  then  the  results  so  obtained  are  usable  at  least-- 
they  may  not  be  •’best."  On  the  other  hand,  when  additional  observations  am 
taken,  leading  to  "over  determination"  and  consequent  contradiction  of  the 
fundamental  properties  of,  01  the  basic  relationships  among  the  quantities 
concerned,  then  the  respective  observations  must  be  regarded  as  contradicting 
one  another.  Whan  this  happens  the  observations  themsulves,  or  values  derived 
from  them,  must  ba  replaced  by  adjusted  values  such  that  all  contradiction 
Is  removed.  "This  Is  a  logical  necessity  ,  since  wc  cannot  accept  for  truth 
that  which  Is  contradictory  or  leads  to  contradictory  results,"  [Chauvenet 
1868,  p.  472.]  1 

2.4.  Scheduling  the  Taking  of  Measurements.  Having  done  v* hat  onr 
can  to  remove  extraneous  sources  of  error,  and  to  make  the  basic  measurements 
as  precise  and  as  free  from  systematic  error  as  possible,  It  Is  frequently  pos¬ 
sible  not  only  to  Increase  the  precision  of  tho  end  results  of  major  Interest 
but  also  to  simultaneously  decrease  their  sensitivity  to  sources  of  poeslble 
systematic  error,  by  careful  scheduling  of  the  measurements  required,  An 
instance  is  provided  by  the  traditional  procedure  for  calibrating  ilquld-ln-glasB 
thermometers  Waldner  and  Dickinson  1907,  p.  7U2,  N°L  1957,  pp.  29-30: 
Swindells  1959,  pp.  11-12  1;  Instead  of  attempting  to  hold  the  temperature  of 
the  comparison  bath  constant,  a  very  difficult  objective  to  achieve,  the  heat 
input  to  the  bath  Is  so  adjusted  that  Its  temperature  is  slowly  Increasing  at  s 
.•ready  rate,  and  then  rearilnqs  of,  say,  four  test  thermometers  end  two  standards 
era  taken  *n  !no''r'.'.auca  with  the  schedule 

SlTlT2T3'14S2S2T4T3T2'ri3l 

the  readings  being  spacod  uniformly  In  time  su  that  the  arithmetic  mean  of  tho 
two  readings  of  any  one  thermometer  will  correspond  to  the  tumperatuie  of  th, 
comparison  bath  at  the  midpoint  of  the  period.  Such  scheduling  of  mnaB'>rem<mt 
tokhig  operations  so  that  the  affects  of  the  specific  types  of  departures  from 
perfect  control  of  conditions  and  procedure  will  have  an  opportunity  to  balance 
out  is  one  of  the  principal  aims  of  the  art  and  science  of  statistical  design  of 
experiments  ,  For  additional  physical  science  examples,  see,  for  Instance, 
Youden  fl951a;  and  1954-19591  . 
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2.5.  Measurement  as  a  Production  Process.  We  may  summarise  our 
discussion  of  measurement  up  to  this  point,  ?s  follows:  Measurement  of  some 
property  of  a  thing  In  practloe  always  takes  the  form  of  a  sequenoe  of  steps  or 
operations  that  yield  as  an  end  result  a  number  that  serves  to  represent  the 
amount  or  quantity  of  some  particular  property  of  a  thtng--a  number  that  Indi¬ 
cates  how  much  of  this  property  the  thing  has,  for  someone  to  use  for  a  epeclflo 
purpose.  The  end  result  may  be  the  outcome  of  u  single  reading  of  an  Instru¬ 
ment,  with  or  without  corrections  for  departures  from  prescribed  conditions. 

More  often  It.  Is  some  kind  of  average  or  adjusted  value,  e.y, ,  the  arithmetic 
mean  of  a  number  of  Independent  determinations  of  the  same  magnitude,  or  the 
final  result  of,  say,  a  least  squares  "reduction"  of  measurements  of  a  number 
of  different  quantities  that  havB  known  relations  to  tha  quantity  of  Interest. 

Measurement  of  some  property  of  a  thing  is  ordinarily  u  repeatable 
operation.  This  Is  cortatnly  the  case  for  the  types  of  measurement  ord;  iarlly 
met  in  the  calibration  of  standards  and  Instruments.  It  Is  Instructive,  therefore, 
to  regard  measurement  as  a  production  process,  the  "product"  being  the  numbers, 
that  Is,  the  measurements  that  It  yields;  and  to  compare  and  contrast  measure¬ 
ment  processes  In  the  laboratory  with  mass  production  prooesses  In  Industry. 

For  the  moment  It  will  suffice  to  note  (a)  that  when  successive  amounts  of 
units  of  "raw  material"  are  processed  by  a  particular  mass  production  process, 
the  output  is  a  series  of  nominally  Identical  Items  of  product--of  the  pertloular 
type  produced  by  the  mass  production  operation,  l,o,,  by  the  method  of  pro¬ 
duction  concerned;  and  (b)  that  when  successive  objects  are  measured  bv  a 
parttculm  measurement  process,  the  Individual  Items  of  "product"  produood 
consist  of  the  numbers  assigned  to  the  respective  objects  to  represent  the 
relative  amounts  that  they  possess  of  the  property  determined  by  th«  method 

2.6.  MflUipdfl-al ■MgamirwmfinLflnd  Jifeajimgiu&lll,  Processes ■  Sp-iol- 
flcatton  of  the  apparatus  and  auxiliary  equipment  to  be  used,  the  operations 
to  be  pi- formed,  the  sequence  In  whlcih  they  are  to  bw  carried  out,  and  th.tf 
conditions  under  which  they  are  respectively  to  bo  carried  out — these  instruction^ 
collectively  serve  to  define  a  method  of  mso  sure  men*.  ■  To  the  extent  that 
corrections  may  be  required  they  are  an  Integral  part  of  measurement.  The 
types  Oi  corrections  that  will  ordinarily  need  to  be  mado,  ami  specific  proce¬ 
dures  for  making  them,  should  be  Included  among  "tha  operations  to  be  per- 
tormnd."  Likewise,  the  essential  adjustments  required  should  bo  noted,  and 
specific  procedures  for  making  them  Incorporated  In  the  specification  of  a 
method  of  measurement. 
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A  measurement  process  Is  the  realization  of  a  method  of  measurement 
In  terms  of  particular  apparatus  and  equipment  of  the  prescribed  Kinds,  parti¬ 
cular  conditions  that  at  best  only  approximate  the  conditions  prescribed,  and 
particular  persons  sb  operators  and  observers  [aSTM  1961,  p.  1758;  Murphy 
1961,  p.  264  1 .  Of  course,  there  will  often  be  a  question  whether  a  particular 
measurement  process  Is  loyal  to  the  method  of  measurement  of  wnioh  it  Is  In¬ 
tended  to  be  a  realization;  or  whether  two  different  measurement  processes  can 
be  considered  to  be  realizations  of  the  same  method  of  measurement. 

To  begin  with,  written  specifications  of  methods  of  measurement  often 
contain  absolutely  precise  Instructions  which,  however,  cannot  be  carried 
out  (repeatedly)  with  complete  exactitude  In  praotlce;  for  example,  "move 
the  two  parallel  cross  hairs  of  the  micrometer  of  the  microscope  until  the 
graduation  lina  of  tha  standard  la  centeiad  between  them.”  The  aocuracy 
with  which  such  Instructions  can  ba  carried  out  In  practice  will  always  depend 
upon  "the  circumstances”;  in  the  case  cited,  on  tha  skill  of  the  opsrator,  ths 
quality  of  the  graduation  line  of  the  standard,  the  quality  of  tha  screw  of  the 
micrometer,  ths  parallelism  of  the  cross  hairs,  sto.  To  the  extant  that  tha 
written  specification  or  a  method  of  measurement  involves  absolutely  precise 
Instructions  that  cannot  be  carried  out  with  complete  exactitude  lu  practice 
there  are  certain  to  be  discrepancies  between  a  method  of  measurement  and 
its  realization  by  a  particular  measurement  process. 

In  addition,  the  specification  of  a  method  of  measurement  often  Includus 
u  number  of  Imprecise  Instructions,  such  ns  "raise  the  temperature  nlov.ly," 

"stir  will  before  taking  a  reading,"  "make  sure  that  the  tubing  is  clean,"  etc. 

Not  only  are  such  Instructions  Inherently  vague,  but  ditto  In  any  given  Instance 
they  must  be  understood  in  e.  m.s  of  t.hei  general  level  of  refinement  characteristic 
of  the  comuxi  la  which  they  occur,  Thus,  "make  sure  that  tha  tubing  is  dean" 
la  not  an  absolutely  definite  instruction;  to  some  people  this  would  mean 
simply  that  the  tubing  should  be  clean  enough  to  drink  liquids  througm  i,: 
some  laboratory  work  It  might  be  Interpreted  to  mean  mechanically  washed 
and  scouted  so  as  to  be  free  from  dirt  and  other  ordinary  solid  matter  (but  not 
cleansed  also  with  chemical  solvents  to  remove  mote  stubborn  contaminants); 
tn  en  advanced  experimental  physicist  It  may  mean  not  merely  mechanics1!/ 
wcr.r.od  and  chemically  cleansed,  but  bIbo  "out  gasses"  by  holng  heated  to 
and  held  at  a  high  temperature,  near  the  softening  point,  for  an  hour  or  so. 

All  will  agree,  I  believe,  that  It  would  bo  exceedingly  difficult  to  make  such 
Instructions  absolutely  dlflnlte  with  a  convenient  number  of  words.  To  the 
extent  that  the  specification  of  a  method  of  measurement  includes  Instructions 
that  are  not  absolutely  definite,  there  will  ba  room  for  differences  between 
measurement  processes  that  are  Intended  to  be  realization  of  tho  very  same 
mot  hod  of  measurement, 
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Recognition  of  the  difficulty  of  achieving  absolute  definiteness  In  the 
specification  of  a  method  of  measurement  does  not  Imply  that  "any  old  sat" 
of  Instructions  will  serve  to  define  a  method  of  measurement.  Quite  the  con¬ 
trary.  To  qualify  as  a  specification  cl  a  method  of  measurement,  a  set  of  In¬ 
structions  must  be  sufficiently  definite  to  Insure  statistical  stability  of  repeated 
measurements  of  a  single  quantity,  that  Is,  derived  measurement  processes 
must  be  capable  of  meeting  the  criteria  of  statistical  control  [shewhart  1939, 
p.  131;  Murphy  1961,  p.  265;  ASTM  1961,  p,  1758  j.  To  elucidation  of  the  mean¬ 
ing  of,  and  need  for  this  requirement  we  now  tuirr. 


3.1.  Requirement  of  Statistical  Control.  The  need  for  attaining  a  de¬ 
gree  of  consistency  among  repeated  measurements  of  a  single  quantity  before 
the  method  of  measurement  concerned  can  be  regarded  as  meaningful  has  cer¬ 
tainly  been  recognized  for  a  long,  long  time.  Thus  Galileo,  describing  his 
famous  experiment  on  the  acceleration  of  gravity  In  which  he  allowed  a  ball  to 
roll  different  distances  down  an  inclined  plane  wrote: 

"...  Sl  lasclava  (oomo  dlco)  scendera  per  11  detto  canals  la  p«Ua, 
notando,  nel  modo  ohe  appresso  dlro,  LI  temp  aha  oonsumava  nello  scorrerlo 
tutt ,  ropucando  11  medeslmo  etto  molte  volte  per  ansiourarsl  bens  della  quan- 
tita  dal  temp,  nel  quale  non  sl  trovava  mat  dtfferenza  ne  anco  della  deolms 
parte  d'una  bettuta  dt  polso.  Fatfa  a  stablllta  preclsamante  tale  operazlone, 
faoeinmo  soender  la  medlstma  palla  solament*  per  la  quarta  parte  della  lunghazza 
dl  eeuo  oanale  .  .,"1  [Galileo  1638,  Third  Day;  Nat'l,  ad.,  p,  213.] 

Something  mnre  thnn  noi'v  “consistency"  is  required,  however,  as 
Shewhart  points  out  eloquently  Ip  his  very  important  chapter  on  "The  Specifica¬ 
tion  of  Accuracy  and  Precision"  jShewhau  1919,  ch  IVJ .  He  begins  by  noting 
that  the  description  given  by  R.  A.  Millikan  [l903,  ppr.  195-196  |  of  a  meii.:d 
for  deicmlnlna  the  surface  tension  X  of  *  liquid  from  measurements  of  tha 


I  anv grateful  to  my  collegue  Ugo  Fano  for  the  following  literal  tianslatlon: 

"...  we  let,  as  I  was  saying,  the  ball  de sound  through  said  channel,  record¬ 
ing,  in  a  manner  presently  to  be  described,  the  time  It  took  in  traversing  it  all, 
repeating  the  same  action  many  times  to  make  really  sure  of  the  magnitude  of 
time,  in  which  one  never  found  a  difference  df  even  a  tenth  of  a  pulsebeat. 
Having  done  and  established  precisely  such  operation,  we  let  the  same  ball 
descend  only  for  the  fourth  part  of  the  length  of  the  same  channel;.  . ." 
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force  of  tension  _F_  of  a  film  of  the  liquid  aontalns  the  following  Instruction 
with  regard  to  the  basic  readings  from  which  measurements  of  £  are  derived: 
"Continue  this  operation  until  a  number  of  consistent  readings  oan  be  obtained." 
Shewhart  then  comments  on  this  as  follows: 

"...  the  text  describing  the  operation  does  not  say  to  carry  out  such 
and  such  physical  operations  and  call  the  result  a  measurement  of  X  Instead, 

It  says  In  effeot  not  to  call  the  result  a  measurement  of  X  until  one  has 
attained  a  certain  degree  of  consistency  among  the  observed  values  of  F_  and 
hence  among  those  of  T.  Although  this  requirement  Is  not  always  axpHoltly 
stated  In  specifications  of  the  operation  of  measurements  as  It  was  hare,  I 
think  It  Is  always  Implied.  Likewise,  I  think  it  Is  always  assumed  that  them 
can  be  too  much  consistency  or  uniformity  among  the  observed  values  as,  for 
example,  If  a  large  number  of  measurements  of  the  surface  tension  of  a  liquid 
were  found  to  be  Identical.  What  is  wanted  but  not  explicitly  daeorlbed  is  a 
specific  kind  and  degree  of  consistency. 

"...  It  should  be  noted  that  the  advice  to  repeat  the  operation  of 
measuring  surface  tension  until  a  number  of  consistent  readings  have  been 
obtained  Is  Indefinite  in  that  It  does  not  Indicate  how  many  readings  shall  ba 
taken  before  applying  a  test  for  consistency,  nor  what  kind  of  test  of  consistency 
Is  to  be  applied  to  the  numbers  or  pointer  readings. . . .  One  of  the  objects  of 
'.hie  chapter  Is  to  see  how  far  one  can  go  toward  improving  this  situation  by 
providing  an  operationally  definite  criterion  that  preliminary  observations 
must  meat  before  they  are  to  be  ooneldered  consistent  in  the  sense  Implied  In 
the  instruction  oltod  above  . 

". ..  Before  dolnq  t n b ,  however,  we  must  give  attention  not  so  much 
In  the  consistency  of  the  n  oossrved  values  already  obtained  by  £  repeti¬ 
tions  of  the  operation  of  measurement  as  we  do  to  the  reproducibility  of  the 
operation  as  determined  by  '.he  numbers  In  the  potentially  tntlntte  se^uC-c* 
corresponding  to  an  Infinite  number  of  repetitions  of  this  operation.  No  one 
would  care  very  muoh  huw  consistent  the  first  ji  preliminary  observation:) 
were  if  nothing  aould  be  validly  Inferred  from  this  as  to  what  future  observa¬ 
tions  would  show  .  Hence,  It  seems  to  me  that  the  characteristics  of  the 
nn’in«rlcal  aspects  of  an  operation  that  Is  of  greatest  practical  Interest  Is  Its 
reproducibility  within  tolerance  limits  throughout  the  infinite  sequence.  Tha 
limit  to  which  we  may  go  in  this  direction  is  to  attain  u  state  of  statistical 
oontroi.  The  attempt  to  attain  a  oertaln  kind  of  oonststenoy  within  the  first 
&  observed  values  is  merely  a  means  of  attaining  reproducibility  wlthlui  limits 
throughout  the  whole  of  the  sequence."  [Shewhart  1939,  pp.  131-132.1 
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The  point  that  Shewhart  makes  forcefully,  and  stresses  repeatedly 
later  in  the  same  chapter,  Is  that  the  first  £  measurements  of  a  given  quan¬ 
tity  generated  by  a  particular  measurement  process  provide  a  logical  basis  for 
predicting  the  behavior  of  further  measurements  of  the  same  quantity  by  the  same 
measurement  process  if  and  only  If  these  q.  measurements  may  be  regarded 
as  a  random  sample  from  a  "population"  or  "universe"  of  all  conceivable 
measurements  of  the  given  quantity  by  the  measurement  prooess  concerned,' 
that  Is,  In  the  language  01  mathematical  statistics,  if  and  only  If  the  £ 
measurements  In  hand  may  be  regarded  as  "observed  values"  of  a  sequence 
of  random  variables  characterized  by  a  probability  llstrlbutlon  Identified  with 
the  measurement  process  concerned,  and  related  through  the  values  of  one  or 
more  of  Its  parameters  to  the  magnitude  of  the  quantity  measured, 

It  should  be  noted  especially  that  nothing  Is  said  about  the  mathema¬ 
tical  form  of  the  uiubabllltv  distribution  of  these  random  variables,  The  Im¬ 
portant  thing  is  that  there  be  one.  W.  Edwards  Dsmlng  has  put  thla  clearly 
and  forcefully  In  these  words: 

"In  applying  statistical  theory,  the  main  consideration  is  not  what 
the  shape  of  the  universe  Is,  but  whether  there  Is  any  universe  et  ell.  No 
universe  can  be  assumed,  nor  . . .  statistical  thoory  . . .  applied  unless  the 
observations  show  stetlsttosl  control.  In  this  state  the  samples  when  cumu¬ 
late  over  a  suitable  Interval  of  time  give  a  distribution  of  s  particular  shape, 
ai.d  this  shape  is  reproduced  hour  after  hour,  day  after  day,  eo  long  aa  the 
process  remains  In  statistical  control-- 1. e. ,  exhibits  the  properties  of  ran¬ 
domness.  In  a  state  of  control,  jj  observations  may  be  regarded  as  a  samp  in 
from  the  universe  of  whatever  shaoe  it  is.  A  big  enough  sample,  nr  enough 
small  sampl«s.  tlio  uiatistinian  to  make  meaningful  and  useful  pre¬ 

dictions  auoui  for., ut!  samples,  This  is  as  much  as  utatlatloal  theory  can  do. 

" . , ,  Very  often  the  experimenter,  Instead  of  rushing  in  to  apply 
[statuucal  methods]  should  be  more  concerned  about  attaining  statlotlcel 
control  and  aeklng  himself  whether  any  predictions  et  all  (the  only  pm  pose 

ft  his  experiment),  by  statistical  theory  or  otherwise,  can  bo  made," 

Doming  1950,  pp.  502-503 , 1 

Shewhart  was  well  aware  of  the  fact  that  from  a  set  of  £  measurements 
In  hand  it  is  not  possible  to  decide  with  absolute  certainty  whether  they  do  or 
do  not  constitute  a  random  sample  from  some  definite  statistical  "population" 
characterized  by  a  probability  distribution.  He,  therefore,  proposed  [shewhart 
1939,  pp.  146-147]  that  In  any  particular  Instance  one  should  "decide  to  act 
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for  the  presont  as  if"2  the  measurements  In  hand  (and  their  immodlate  successors) 
were  a  simple  random  sample  from  a  definite  statistical  population--!. a. ,  In 
the  language  of  mathematical  statistics,  were  "observed  values"  of  Independent 
Identically  distributed  random  vartablea--onlv  If  the  measurements  In  hand  met 
the  requirements  of  the  smell- samples  version  of  Criterion  I  of  his  previous 
book  [shewhart  1931,  pp.  309-310  ]  and  of  certaLn  additional  test  of  randomness 
that  he  desorlbed  explicitly  for  the  first  time  In  his  contribution  to  the  Univer¬ 
sity  of  Pennsylvania  Bicentennial  Ccpnference  In  September  1040  [showhart, 

1941  j .  In  other  words,  Shewhart  proposed  that  one  should  consider  a  measure¬ 
ment  process  to  be--t ,  should  "decide  to  act  for  the  present  as  if"  the 
process  were-- In  a  state  of  (simple)  statistical  control, only  If  the  measure¬ 
ments  in  hand  show  no  evidence  of  laok  of  statistical  control  when  analysed 
for  randomness  In  the  order  In  which  they  were  taken  by  the  control  chart 
techniques  for  averages  and  standard  deviations  that  he  found  so  valuable  in 
industrial  process  control  and  by  certain  additional  tests  for  randomness  based 
on  "runs  above  and  beiow  average"  and  "runs  up  and  down."'* 

Simpson3  did  not  prove  that  taking  of  the  Arithmetic  Mean  was  the  best 
thing  to  do  but  merely  that  It  is  good.  However,  In  accomplishing  this  goal 
he  did  something  much  moru  Important:  he  took  the  bold  step  of  regarding 
errors  of  measurement,  not  as  unique  unrelated  magnitudes  unamenable  to 


ZThls  vary  explicit  phraseology  is  due  to  John  W.  Tukey  [  I960,  p.  424  j. 
^Thomas  Simpson,  In  his  now  famous  letter  [Simpson  1755]  to  the  President  of 
H'b  Royal  Society  of  London  "on  the  Advantage  of  taking  tht»  Moan  ut  a  Number 
of  Observations,  m  pi^tica!  Astronomy,"  was  the  firet  to  consider  reposted 
measurements  ul  o  single  quantity  by  a  given  measurement  process  as  observed 
values  of  Independent  random  variables  having  the  same  probability  distribu¬ 
tion.  His  conclusion  is  of  Interest  In  Iteolf: 

"Upon  the  whole  of  which  It  appears,  that  the  taking  uf  the  Mean  of 
a  number  of  observations,  greatly  diminishes  the  chances  for  all  the  samller 
errors,  and  outs  off  almost  all  possibility  of  any  great  ones:  which  lust  con¬ 
sideration,  alone,  suem  sufficient  to  recommend  Lhe  use  of  the  met, hud,  not 
only  to  astronomers,  but  to  all  others  concerned  In  making  of  experiments  of 
any  kind  (to  which  the  above  reasoning  Is  equally  applicable).  And  the  more 
observations  or  experiments  there  are  made,  the  less  will  the  conclusion  be 
liable  to  err,  provided  they  admit  of  being  repeated  under  the  same  circum¬ 
stances," 


484 


Design  of  Experiments 


mathematical  analysis,  but  as  distributed  in  accordance  with  a  probability 
distribution  that  was  an  Intrinsic  property  of  the  measurement  process  itself. 

Ha  thus  opened  th®  w*y  to  a  mathematical  theory  of  measurement  based  on  thn 
mathematical  theory  of  probability!  and,  In  particular,  to  the  formulation  and 
development  of  the  Method  of  Least  Squares  in  essentially  Its  present  day  form 
by  Gauss  (1809.  1821)  and  Laplace  (1812), 

"Student"  (William  Sealy  Gusset  1876-19371  pioneer  statistical  con¬ 
sultant  und  "fathnr"  of  the  "theory  of  small  samples,"  wan  certainly  among 
the  first  to  stress  the  Importance  of  randomness  In  measurement  and  experi¬ 
mentation.  Thus,  he  began  his  revolutionary  1908  paper  on  "The  probable 
error  of  a  mean"  with  these  remarks; 

"Any  experiment  may  be  regarded  as  forming  an  Individual  of  a  'popu¬ 
lation'  of  experiments  which  might,  be  performed  under  tha  same  conditions. 

A  nerlaa  ol  experiments  is  a  sample  drawn  from  this  population. 

"Now  any  series  of  experiments  Is  only  of  value  In  so  far  as  It  enables 
us  to  form  a  judgment  as  to  the  statistical  constants  of  tho  population  to  wruch 
the  experiments  belong."  £  Student  1908.  p.  I.j 

Nona  of  these  writers,  not  any  of  their  contemporaries,  however,  pro- 
v'd'”1  "un  operationally  definite  criterion  that  preliminary  observations  must 
meet"  before  we  tuke  It  upon  ourselves  "*o  nc'  for  the  present  as  if"  they  and 
their  Immediate  successors  were  random  samplns  iiom  a  "population"  or  "uni¬ 
verse"  of  all  concnlvable  measurements  of  the  given  quantity  by  the  measure¬ 
ment!,  of  tha  given  quantity  by  thn  measurement  piocess  concerned.  Provision 
of  much  n  c-.luiion  Is  bnewnait 1 i»  major  contribution . 

Experience  shows  that  In  the  case  of  measurement  prouusso®  the  Ideal 
oi  stilct  statistical  control  that  Shewhart  prescribes  Is  usually  very  dlfticuil  ’n 
attain,  Just  os  In  the  oe*p  of  industilal  producuon  processes,  Indeed,  mans’ 
muuuuremerit  processes  simply  do  not  and,  It  would  cacm,  cannot  be  made  to 
conform  to  this  Ideal  of  producing  successive  mnasi  rements  of  a  single  quin- 
1  It y  rent  can  bo  considered  to  be  "observed  valuer"  of  Independent  Idontioe.i/ 
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distributed  random  variables. 4  The  nature  of  the  "tiouble"  was  stated  suoctnotly 
by  Student  in  1917  when,  speaking  of  physical  and  ohemloal  determinations, 
he  wrote: 

"After  considerable  experience  l  have  not.  encountered  any  determina¬ 
tion  whtah  Is  not  influenced  by  the  date  on  which  It  is  made;  from  this  It  follows 
that  a  number  of  determinations  of  the  some  thing  made  on  the  same  day  are 
likely  to  lie  more  closely  together  than  If  the  repetitions  had  been  made  on 
different  days."  ]  Student  1917,  p.  415. j 

In  other  words,  production  of  measurements  seems  to  be  like  the  production 
of  paint;  and  just  as  In  the  case  of  paint,  if  one  must  cover  a  large  surface 
all  of  which  Is  visible  simultaneously,  one  will  do  well  to  use  pelnt  all  from 
the  same  batch,  so  In  the  case  of  measurements,  It  a  scientist  or  metroloylst 
"wishes  to  Impress  his  clients"  he  Will  "arrange  to  do  repetition  analyses  as 
nearly  as  possible  at  tho  Bame  time."  ^Student  1927,  p.  155, j 

Fortunately,  just  as  one  may  blend  paint  from  several  batches  to  ob¬ 
tain  e  morn  uniform  color,  and  one  which  Is,  presumable,  closer  to  the  "pro¬ 
cess  average  ,"  so  also  may  a  solentlst  or  metrolggist  "if  ha  wishes  to  dimin¬ 
ish  his  real  error,  ,  . .  separate]  his  measurements  J  by  as  wide  an  Interval  of 
time  as  possible"  [  Student,  loo.  olt.j  and  then  take  an  appropriate  average  of 
diem  as  hi s  determination.  Consequently,  if  wo  uio  to  permit  such  averaging 
■as  on  allowable  step  in  a  fully  specified  measurement  process  (s»o  see.  2.6 
shove),  then  we  are  obliged  to  recognize  both  wlthln-dny  and  bi.tweon-day 
components  of  variation,  and  accept  such  a  complex  measurement,  process  as 
b.,;ng  In  a  state  of  statistical  control  overall,  or  as  we  shall  say,  In  a  Mate 
el  COMPLEX  statistic? '  uu.diul.  when  the  component*  ol  withm-day  and  between 
day  variation  «:u  both  in  a  state  of  slaiistiaal  control  In  Shrwhart's  strict 
sense,  which  we  shall  term  SIMPLl  statistical  control,  In  more  complex  situa¬ 
tions,  one  may  be  obliged  to  recognize  more  than  two  "layers"  of  varied,  o 


4j, coking  at  the  matter  from  a  fundamental  viewpoint,  perhaps  we  should  say, 
not  that  Shewhart's  Ideal  of  strict  statistical  control  1b  unattainable  In  t!,« 
case  of  such  measurement  processes,  but  rather  that  the  degree  ot  approxima¬ 
tion  to  this  Ideal  can  be  made  as  close  as  one  chooses,  If  one  Le  willing  to 
pay  Hie  price.  In  other  words,  how  close  one  chooses  to  bring  a  measurement 
process  to  the  ideal  of  strict  statistical  control  Is,  in  any  given  instance, 
basically  an  economic  matter,  taking  into  account,  of  oourse,  not  only  the 
immediate  purpose(s)  for  which  the  measurements  are  intended  but  also  the 
other  ubsb  to  which  they  may  be  put.  (Compare  Simon  1 1946,  p.  566  jand 
Etoenhait  f  1952,  p.  S54|  ).  1  1 
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and,  sometimes,  move  than  e  single  component  of  variation  within  e  given 
"layer . " 


Adopting  this  more  general  concept  of  statistical  control,  R.  B,  Murphy 
of  the  Bell  Telephone  Laboratories  In  hla  essay  "On  the  Meaning  of  Precision 
and  Accuracy"  [  Murphy  1961  ,  published  In  advance  of  the  Issuance  by  the 
American  Society  for  Testing  and  Meterlclu  of  Its  Tentative  Reoommended 
Practice  with  respect  to  the  "Use  of  the  Terms  Precision  and  Accuracy  as 
Applied  to  Measurement  of  a  Property  of  a  Material"  [ASTM  19 61  j  ,  remarks: 


"Following  through  with  this  Una  of  thought  borrowed  from  quality  con¬ 
trol,  we  shall  ado  a  requirement  that  an  effort  to  follow  a  test  method  ought 
not  to  be  known  as  a  measurement  process  unless  it  is  capable  of  statistical 
control.  n«nahili»y  of  control  means  that  either  the  measurements  are  the  pro¬ 
duct  of  an  Identifiable  statistical  universe  or  an  orderly  arrey  of  such  universes 
or,  if  not,  the  physical  oauses  pioventlng  such  Identification  may  thamselvei 
be  identified  end,  If  desired,  isolated  and  suppressed,  Incapability  of  control 
implies  that  the  results  of  measurement  are  not  to  be  trusted  as  Indications 
of  the  physical  property  at  hanch-'.n  short,  wa  are  not  In  any  verifiable  Sanaa 
measuring  anything. . . .  Without  this  limitation  of  tha  notion  of  maasurament 
process,  one  Is  unable  to  go  on  to  give  meaning  to  those  statistical  measures 
wi.'ch  are  basic  to  any  dlsousslon  of  precision  and  accuracy,"  [Murphy  1961, 

PD.  264- 265 .]  L 


3.2.  Paitulata  of-.Maa»urament  and  the  Gonoupt  of  a  Llmitlnu  Mean. 

A  conspicuous  characteristic  of  measurement  Is  disagreement  of  repeated 
meos  irements  of  the  same  quantity.  Experience  show*  that,  when  high  accu¬ 
racy  i»  sought,  repeated  measurements  of  the  same  quantity  oy  a  particular 
measurement  procuss  does  not  yl«ld  uniformly  the  uam«  number.  Wa  qx 
plats  these  discordances  by  saying  that  the  individual  measurement*  ere 


’The  qualification  "when  high  accuracy  Is  sought"  is  ensontlel;  lot  If  using 
an  ordinary  two-pan  chemic  al  balances  we  measure  and  record  the  mass  of  a 
small  metallic  objact  only  to  the  nearest  ytum,  then  w»  would  expect  ail  jf 
our  measutemanta  to  be  tho  ramn--axcept  In  the  equivocal  case  of  a  mess 
equal,  or  vary  nearly  equal,  to  an  odd  multiple  of  1/2  g,  and  such  equivocal 
cases  can  be  resolved  easily  by  adding  a  1/2  g  masa  to  one  pan.  Full  accor¬ 
dance  of  measurements  clearly  cannot  be  taken  as  incontestable  evidence  of 
hlcjh  accuracy;  but  rather  should  bo  regarded  as  evidence  of  limited  accuracy, 
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affani  jU  uv  errors ,  whioh  we  Inteipret  to  be  the  manifestations  of  variation* 

In  the  axeoutlon  of  tne  process  of  measurement  resulting  from  "the  Imperfec¬ 
tions  of  lnatrumenta,  and  of  organa  of  aenae,"  and  from  the  difficulty  of  achiev¬ 
ing  (or  even  apeoLfytng  with  a  convenient  number  of  worda)  the  Ideal  of  per- 
feot  control  of  conditions  end  procedure. 

Thla  "cusaedneas  of  measurements"  bringa  ua  face  to  face  with  a  funda¬ 
mental  question!  In  what  aenae  oen  we  aay  that  the  meaaurementa  yielded  by 
a  particular  measurement  proceaa  aerve  to  determine  a  unique  magnitude,  when 
experience  shown  that  repeated  measurement  of  a  alngle  quantity  by  thla  pro¬ 
ceaa  yields  a  sequence  of  nonldentloal  numbers.  What  la  the  value  thus 
determined? 

The  answer  takes  the  form  of  a  postulate  about  maaaurement  prooeaaea 
that  hea  been  expressed  by  N,  Ernest  Dorsey,  as  follows! 

"The  moan  of  a  family  of  measurements—  of  a  number  of  measurements 
for  a  given  quantity  carried  out  by  the  same  apparatus,  prooedurs  and  observer— 
approaohea  a  definite  value  as  the  number  of  measurements  la  Indefinitely 
Increased.  Otherwise,  they  oould  not  properly  be  called  measurements  of  a 
given  quantity.  In  the  theory  of  errors,  this  limiting  mean  Is  frequently  called 
the  'true'  value,  although  It  bears  no  necessary  relation  to  the  true  quaesltum, 
to  the  actual  value  of  the  quantity  that  the  observer  doslras  to  moasuro.  This 
has  often  confused  the  unwary.  Let  us  call  It  the  limiting  mean."  [Dorsey 
1944,  p.  4,-  Dorsey  and  Clsenhart  1953,  p.  103,1 

In  my  lectures  at  the  National  Bureau  of  Standards,  and  elsewhere.  I 
:>?tve  termed  this— or  rather  e  slightly  rephrased  version  nt  U--  the  Postulate 
of  Measurement .  A  mathematical  basis  for  it  la  provided  by  the  Strong  Law 
of  Large  Numbers,  a  theorem  in  the  mathematical  thaory  of  probability  dis¬ 
covered  during  the  present  century.  See,  for  example.  Feller  [i«C7,  pn.  243- 
245,  374  J,  Gnedenko  |J962,  pp.  241-2491,  or  Porsen f  I960,  ph  420  j. 

Needless  to  say,  by  a  "family  of  measuiemente"  Dorsey  means,  not  a 
succession  of  "raw"  readings,  but  rather  a  succession  of  adjusted  or  ooTeotad 
'•alue i  whioh,  by  virtue  of  adjustment  or  correction,  can  rightfully,  bv  con¬ 
sidered  to  be  determinations  of  a  single  magnitude. 

a.  Mathematical  Formulation 

The  foregoing  con  be  expressed  mathematically  as  follows!  on  some 
particular  occasion,  say  the  i)h,  we  may  take  a  number  of  successive  measure¬ 
ments  of  a  single  quantity  by  a  given  measurement  process  under  certain 
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specified  clmumstances.  Let 


denote  the  sequence  of  measurements  so  generated.  Conceptually  at  least, 
this  sequence  could  be  continued  indefinitely.  Likewise,  on  different  occasions 
we  might  start  a  new  sequence ,  using  the  same  measurement  procedure  and 
applying  It  to  measurement  of  the  same  quantity  under  the  earns  fixed  set  uf 
circumstance* .  Each  such  fresh  "start"  would  correspond  to  a  different  value 
of  l.  .  If,  for  example,  the  meaeuremen1  process  concerned  La  statistically 
stable  In  the  sense  of  being  in  a  state  of  statistical  control  aa  defined  by 
Shewhart  [  19.19J  ,  than  the  Strong  Law  of  Large  Humbert  will  be  applicable 
and  we  rr.Ay  expect  the  itaquenoe  of  cumulative  arithmetic  means  on  the 
occasion,  namely, 

(2)  Kin  ■  (xu  +  xl2  ♦  ...+  Kln)/n,  (n«  1,  2,...), 

to  converge  to/*  ,  a  number  that  constitutes  the  limiting  mean  associated 
with  the  quantity  measured  by  thle  measurement  process  under  the  circum¬ 
stances  concerned,  but  Independent  of  the  "oooaelon,"  that  it,  Independent 
□f  the  value  of  The  Strong  Law  of  Large  Numbers  doss  not  guersntee 
that  !l."«  sequence  (2)  for  a  particular  value  of  “i."  will  converge  to/4  as  the 
number  of  observation!  jj  on  this  occasion  tende  to  Infinity,  but  simply  states 
that  among  tho  family  ot  such  sequences  corresponding  to  a  large  number  of 
different  starts,  (L  ■  1-  2,  . the  Instances  of  nonconvargenoa  to  A  will 
be  rem  -xoeotlons ,  In  other  words,  tf  the  measurement  process  with  which 
one  Is,  concerned  satisfies  the  condition!  for  validity  of  the  Strong  T.uw  of 
Largo  Numbers,  then  in  practice  one  Is  almost  certain  to  be  working  with  a 
"good"  sequence— one  for  which  (2!  would  oonverge  to/*  If  the  number  of 
observations  wore  continued  Indefinitely— but  "bad"  occasions  oan  occur, 
though  inrely.  Thus,  the  Postulate  cif  Measurement  expresses  something 
batter  than  an  "uu-the-averago"  property— It  expresses  an  "in-almosv-hli- 
cas*s"  property.  Furthermore,  this  limiting  mean/*  ,  the  value  of  whlon 
each  individual  meaaurament  x  is  trying  to  express,  non  be  regarded  not 
only  us  the,  mean  or  "center  of  gravity"  of  the  Infinite  conceptual  populeth  n 
ot  all  measurements  x  that  might  conceivably  be  generated  by  the  msasure- 
munt  proceaa  concerned  under  the.  specified  olrcumstancss,  but  also  as  the 
value  of  the  quantity  concerned  as  determined  bv  this  measurement  process, 


o,  Aim  of  the  Postulate 

The  sole  aim  of  the  Postulate  of  Measurement  Is  axiomatic  acceptance 
r>!  tho  existence  of  a  limit  approached  by  the  arithmetic  mean  of  a  finite  numbsr 
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n  of  measurements  generated  by  any  measurement  process  as  ji-»oo  .  It  says 
nothing  about  how  the  "best"  estimate  of  this  limiting  mean  la  to  be  obtained 
from  a  Unite  number  of  such  observations.  The  Postulate  Is  an  answer  to  the 
need  of  the  practical  man  for  a  Justification  of  hts  cesite  to  consider  the  se~ 
quence  of  nonldenttcal  numbers  that  he  obtains  when  he  attempts  to  measure 
a  quantity  "by  the  same  method  under  like  circumstances"  as  pertaining  to 
a  single  magnitude,  in  spite  of  the  evident  discordance  ot  its  elements.  The 
Postulate  alms  to  satisfy  this  need  by  telling  him  that  If  he  were  to  continue 
taking  more  and  still  more  measurements  on  this  quantity  "by  the  same  method 
under  like  circumstances"  ad  infinitum,  and  wero  to  calculate  their  cumula¬ 
tive  arithmetic  means  at  successive  stages  of  this  undertaking,  then  he  would 
find  that  the  successive  terms  of  this  sequence  of  cumulative  arithmetic  means 
would  settle  down  to  a  narrower  and  ever  narrower  neighborhood  of  some  de¬ 
finite  number  which  he  could  then  accept  as  the  value  of  the  magnitude  that 
his  first  few  measurements  wero  striving  to  express. 

c.  Importance  of  Limiting  Mean 

The  concept  of  a  limiting  mean  associated  with  the  measurement  of  a 
given  quantity  by  a  particular  measurement  process  that  Is  In  a  Btate  of  JMi.tr 
tlcal  control  is  important  because  by  means  of  statistical  methods  based  on 
'.he  mathematical  theory  of  probability  we  can  make  quantitative  inferential 
statements,  with  known  chances  of  error,  about  the  maynllude  of  this  limiting 
mean  from  a  set  of  measurements  of  the  given  quantity  by  the  measurement 
process  concerned.  The  magnitude  of  Ihe  limiting  moan  associated  wun  the 
measurement  of  a  given  quantity  by  a  particular  measurement  process  must 
he  carefully  distinguished  from  the  true  magnitude  of  the  quantity  measured, 
about  which  we  may  be  temr*°d  to  make  similar  infcientlal  statements  In¬ 
sofar  as  we  make  statistical  inferences  from  a  set  of  measurements,  we  make 
idem  with  respect  to  a  property  of  the  measurement  process  lnv',’ved  under 
•  bo  circumstances  concerned.  The  step  from  quantitative  Inferential  suni-- 
irents  about  the  limiting  mean  assoclatnd  with  the  measurement  of  a  given 
quantity  by  a  particular  measurement  piocess,  to  quantitative  utatementa 
about  the  true  magnitude  of  the  quantity  concerned,  may  be  based  on  sub¬ 
ject  matter  knowledge  and  skill,  general  information  and  lntultlon--"'ut  not 
on  statistical  methodology,  (Compare  Cochran,  Mosteller,  and  Tukey  [  19 53  , 
pp.  632-693  ]  .)  1 
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3.3.  Definition  of  the  Error  of  a  Measurement,  and  of  the  Systematic 
Error.  Precision,  and  Accuracy  of  a  Measurement  Process. 

u.  Error  of  a  Single  Measurement  or  Adjusted  Value 

The  error  of  any  measurement  of  a  particular  quantity  Is,  by  defini¬ 
tion,  the  difference  between  the  measurement  concerned  and  the  true  value 
of  the  magnitude  of  this  quantity  taken  positive  or  negative  accordingly  as 
the  measurement  Is  greater  or  less  than  the  true  value.  In  other  words,  If 
x  denotes  a  single  measurement  of  a  quantity,  or  an  adjusted  value  derived 
from  a  specific  set  of  Individual  measurements,  and  X  Is  the  true  value  of 
the  magnitude  of  the  quantity  concerned,  then,  by  definition, 

the  error  of  x  as  a  measurement  of  X  *  x  -T. 

The  error  of  any  particular  measurement  or  adjusted  value,  x,  Is, 
therefore,  a  fixed  number.  Th*  numerical  magnitude  and  algn  of  this  number 
will  ordinarily  be  unknown  and  unknowable,  because  the  true  value  of  the 
magnitude  of  the  quantity  concerned  Ls  ordinarily  unknown  and  unknowable  . 
Limits  to  the  error  of  a  single  measurement  or  adjusted  value  may,  however, 
be  Inferred  from  (a)  the  precision,  and  (b)  bounds  on  the  systematic  error, 
oi  the  measurement  process  by  which  It  was  produced- -but  not  without  risk 
t/  lifting  Incorrect,  because,  quite  apart  from  the  Inexactness  with  which 
bounds  are  commonly  placed  on  the  systematic  error  of  a  measurement 
process,  such  limits  are  applicable  to  the  error  of  a  singU  measurement  or 
adjured  value,  not  au  a  unique  Individual  outcome,  but  only  as  a  typical  case 
of  the  errors  characteristic  of  measurements  of  the  same  quantity  that  might 
have  been,  or  might  be,  yielded  by  the  same  measurement  process  under  the 
same  conditions . 


b.  Systematic  Error  of  a  Measurement  Process 

When  the  limiting  meon^  associated  with  measurement  of  the  magni¬ 
tude  of  a  quantity  by  a  particular  measurement  process  doc3  not  agree  with  the 
true  »alua  r  of  the  magnitude  concerned,  the  measurement  process  Ls  sa'd  to 
have  a  systematic  error,  or  bias,  of  magnitude  jjl  -  X- 

The  systematic  error  of  a  measurement  process  will  ordinarily  have 
both  constant. and  variable  components.  Consider,  for  example,  measurement 
of  the  distance  between  two  points  by  means  of  a  graduated  metal  tape 
[Holman  1892 ,  p.  9  1.  Possible  causes  of  systematic  error  that  Immediately 
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come  to  mind  are! 

(1)  Mistakes  In  numbering  the  scale  divisions  of  the  tape; 

(2)  Irregular  spacing  of  the  divisions  of  the  tapo; 

(3)  sag  of  tape; 

(4)  stretch  of  tape; 

(5)  temperature  not  that  for  which  the  tape  was  calibrated. 

For  env  single  dl stance,  the  effects  of  (1)  and  (2)  will  be  constant;  and  the 
effects  of  (3)  and  (4)  will  undoubtedly  each  contain  a  constant  component 
characteristic  of  .he  distance  concerned.  Some  of  these  effects  will  be  of 
one  sign,  some  of  the  other,  and  their  algebraic  sum  will  determine  the  con¬ 
stant  error  of  this  measurement  process  with  respect  to  the  particular  distance 
concerned.  Furtheimota,  the  ’’constant  error"  of  this  measurement  prooeBii 
will  be  different  (at  least,  conceptually)  for  different  distances  measured. 

In  the  case  of  repeated  measurement  of  a  single  distance,  the  affect 
of  (5),  and  at  least  portions  of  the  effeots  of  (3)  and  (4),  may  be  expeoted 
to  vary  from  one  "occasion"  to  the  next  (e.g. ,  from  day  to  day),  thus  con¬ 
tributing  variable  components  to  the  systematic  error  of  the  process. 

A  large  fraction  of  the  variable  contributions  of  (3)  and  (4)  could, 
and  in  practice  no  doubt  would,  be  removed  by  stretching  the  tape  by  a 
spring  balance  or  other  means  so  that  It  Is  always  under  the  same  tension. 

The  stretch  corresponding  to  a  particular  distance  would  then  be  nearly 
the  same  at  all  times,  and  a  fixed  correction  could  ba  made  for  most  cf  tt>o  lay 
corresponding  to  this  dl  stereo.  Furthermore,  the  effect  of  (3)  uuuid,  and  In 
practice  pioLdbly  would,  be.  reduced  by  determining  the  temperature  cf  the 
tape  at  various  points  alor.s  its  length  end  applying  a  temperature  correction. 

3y  comparison  of  the  tape  with  a  standard,  the  error  arising  fro,,.  (’  1  could  bo 
eliminated  entirely,  and  corrections  determined  as  a  basis  for  eliminating,  ~r 
at  least,  reducing  the  effect  of  (2), 

As  In  the  foregoing  example  there  arc  usually  certain  obvious 
sources  of  systematic  error,  Unfortunately,  there  are  generally  edih- 
tlonal  sources  of  systematic  error,  thu  detection,  diagnosis,  and  eradication 
of  which  call  for  much  patience  and  acumen  on  the  part  ot  the  observer.  The 
work  Involved  In  their  detection,  diagnosis,  and  eradication  often  far  exceeds 
that  cf  taking  the  final  measurements,  and  Is  sometimes  discouraging  to  the 
expeiienced  observer  as  well  as  to  the  beginner,  Fortunately,  there  are  various 
statistical  tools  that  aro  helpful  in  this  connection,  and  Olmsteud  flD52l 
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has  found  that  of  these  the  two  most  effective  and  universally  useful  are  the 
average  ( st  )  and  range  (  R  )  charts  of  Industrial  quality  control.  (For  details 
on  the  construction  and  use  of  x-  and  R-charts,  see,  for  example,  the  A  STM 
Manual  on  Quality  Control  of  Materia  1st ASTM  1951,  pp,  61-63  and  p.  83j  ; 
or  American  Standards  Zl. 2-1958  and  Zll.3-1958^ASA  1958b,  ASA  1958a j.) 

c,  Concept  of  True  Value 

In  the  foregoing  we  have  defined  the  error  of  a  measurement  x  to  be 
the  difference  x  -  T  between  the  measurement  and  the  true  value  r  of  the 
magnitude  of  the  quantity  concerned:  and  the  svatematla  error  ■  or  bias,  of  a 
measurement  process  an  the  difference  jx  -  t  between  the  limiting  mean 
associated  with  the  measurement  of  a  particular  quantity  by  the  measurement 
process  concerned,  and  the  true  value  Y  of  the  magnitude  of  this  quantity, 

This  Immediately  raises  the  question:  Just  how  la  the  "true  value"  of  the 
magnitude  of  a  particular  property  of  some  thing  defined?  In  the  final  analy¬ 
sis,  the  “true  value"  of  the  magnitude  of  a  quantity  la  defined  by  agreement 
among  experts  on  an  exemplar  method  for  the  measurement  of  Its  magnitude-  - 
It  is  the  limiting  mean  of  a  conceptual  exemplar  process  that  Is  an  ideal 
realization  of  the  agreed-upon  exemplar  method.  And  the  refinement  to  which 
one  should  go  in  specifying  the  exemplar  process  will  depend  on  the  purposes 
for  .viiioh  e  deteimlnatlon  of  the  magnitude  of  the  quantity  concerned  Is  needed- 
-r.ot  I’tst  the  immediate  purpose  for  wnloh  measurements  tiro  to  b«s  taken  but 
also  the  other  uses  to  which  these  measurements  ,  or  a  final  adjusted  value 
derived  therefrom,  may  poaslbty  be  put. 

Consider,  for  example,  the  "true  value”  of  the  length  of  a  particular 
gage  block.  In  our  mlnUo  «ve  envisage  the  gage  block  as  a  '•octangular  parallel¬ 
epiped,  and  Its  length  Is,  of  course,  the  distance  batween  Its  two  "end" 
faces.  But  It  Ir  practically  certain  that  the  particular  gage  block  in  question 
Is  not  an  exact  rectangular  parallelepiped;  and  that  Its  two  end  faoee  are  ..?* 
planer ,  nor  even  absolutely  smooth  surfaces.  Shall  we  define  the  "true 
length"  of  this  gage  block  to  be  the  dtstance  between  the  "tops"  of  the 
highest  “mountains"  at  each  tnd,  l,e.,  the  distance  between  the  two  "outer¬ 
most  points"  at  each  end?  If  so,  is  this  distance  to  be  measured  diagonally, 
if  necessary,  or  parallel  to  the  "lengthwise  axis"  of  the  gage  block?  t.  ti.a 
latter,  then  we  have  the  problem  of  how  this  "length-wise  axis"  Is  to  be  de¬ 
fined,  especially  In  the  caso  of  a  thin  gage  block  whose  leppth  corresponds 
to  what  would  ordinarily  be  considered  to  be  its  thickness.  Or  shall  we  be, 
perhaps,  more  sophisticated,  and  envisage  a  "mean  piane"  at  each  end, 
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whl^h  ir.  general  will  not  be  parallel  to  each  other,  and  define  the 
;ogth  of  this  gage  block  to  be  the  dlstanoe  between  two  particular  points 
on  these  planes.  If  we  choose  the  "outermost  points"  we  again  have  the 
problem  of  the  direction  In  which  the  distance  Is  to  be  measured,  Al¬ 
ternatively,  we  mLght  define  the  length  of  this  gage  block  to  be  the  dis¬ 
tance  between  two  strictly  parallel  and  conceptually  perfect  optical  flats 
"just  touching"  the  gage  block  at  each  end.  If  so,  then  Lb  the  "true  dlstanoe" 
between  these  flats  defined  In  terms  of  wavelengths  of  light  via  the  tech¬ 
niques  of  optical  Interferometry  the  "true  length"  of  the  gage  block  appro¬ 
priate  to  the  purposes  for  which  the  gage  block  le  to  be  used,  namely,  to 
calibrate  gages  and  to  determine  the  lengths  of  other  objects  by  mschant- 
Sftl. comparisons?  Furthermore,  It  is  clear,  that  the  intrinsic  difficulty  of 
defining  the  “true  value"  of  the  length  of  a  particular  gage  blook  Is  not 
eliminated  If,  instead,  we  undertake  to  define  the  "true  value"  of  the 
difference  In  length  of  two  particular  gage  blooke,  one  of  whloh  la  a 
standard,  the  accepted  value  of  those  length  Is,  say,  m  mlorolnohee 
axaotlv.  by  Industry,  national  or  International  agreement. 

Slmllat  difficulties  arise,  of  course,  In  the  definition  of  the  "trua 
value"  of  the  mass  of  a  mass  standard,  one  of  wntch  has  been  resolved 
by  International  agreement.  In  defining  the  "true  value"  of  the  mass 
of  a  particular  metallic  mass  standard,  shall  Hut  mass  of  this  particular 
standard  be  envisaged  as  the  mass  of  Its  metallic  substance  alone,  rela¬ 
tive  to  the  International  Prototype  Kilogram,  or  as  the  mass  of  Its  metallic 
substance  plus  the  maes  of  the  air  and  water  vapor  absorbed  upon  1H 
surface  under  standard  conditions?  The  difference  amounts  to  about  45 
/Ag  in  the  oase  of  e  platinum-iridium  standard  kilogram,  and  becomes 
critical  In  the  case  of  500  mg  standards,  the  mass  of  n  muse  standard  Is, 
therefor*,  specified  in  mi'^ourement  science  to  be  the  mass  of  the  metallic 
substance  ut  the  standard  plus  the  moss  of  the  average  volums  of  air  ab¬ 
sorbed  upon  Its  sur  face  under  standard  conditions.  Definition  uf  *h« 

"true  value"  of  the  mass  of  a  mass  standard,  and  a  fortiori,  of  the 
different  In  mass  of  two  mass  standards  te,  therefore,  a  vtwy  complex, 
matter. 

W.  Edwards  Demlng  uses  the  expression" preferred  procedure"  or  what 
we  have  termed  an  "exemplar  method,"  ana  very  sagely  remarks  that  "a 
preferred  procedure  Is  distinguished  by  the  fact  that  It  supposedly  gives 
or  would  give  results  nearest  to  what  are  needed  for  a  particular  end;  and 
also  by  the  fact  that  It  Is  more  expansive  or  more  time  consuming,  or 
even  Impossible  to  carry  out, "  adding  that  "as  a  preferred  procedure  Is 
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always  aul>]..at  to  modification  or  obsolescence,  we  are  forced  tu  conclude 
that  neither  the  accuracy  nor  the  bias  of  anv  procedure  can  ever  be  known  In 
a  logical  apnie,"  [Penning  1950.  pp.  15-17. j 

It  should  be  evident  from  the  foregoing  that  the  "true  value"  of  the  magni¬ 
tude  of  some  property  of  a  thing  or  syc*em  cannot  be  defined  with  complete 
absolute  exactitude. 

As  Cantus  J.  Kayser  has  remarked.  "Absolute  oertelnty  Is  a  privilege 
of  uneducated, mlnds--and  fanatics.  It  Is,  for  scientific  folk,  an  unattain¬ 
able  Ideal."  [Kayser  1922,  p.  120,1  The  degree  of  refinement  to  which  one 
will  or  ought  to  go  In  a  particular  instance  will  depend  on  the  uses  for  which 
knowledge  of  the  magnitude  of  the  property  concerned  is  needed.  The  "true 
value"  of  the  length  of  a  piece  of  doth  in  everyday  commerce  Is  certainly  a 
fuzzy  concept.  "Certainly  we  are  not  going  to  specify  that  the  alot.h  shall 
he  measured  while  suspended  horlsontaily  under  a  tonslon  of  x  pounds,  at 
on  ambient  temperature  of  y  degrees  and  a  rslatlve  humidity  of  e  percent" 

| Simon  1946,  p.  654  1 .  On  ths  other  hand,  a  moderate  degree  of  refinement 
fs  neoessary  In  defining  the  "true  length"  and  "true  width"  of  the  recessed 
u.oa  In  a  window  sash  to  which  a  pane  of  gloss  Is  to  bs  fitted.  Oonalderably 
greater  refinement  La  needed  In  t.,e  definition  of  the  "true  value"  of  the  length 
of  a  gage  block,  of  the  mass  of  a  mass  stsndard  or  of  the  frequency  of  a 
CteVicnny  standard- -and  In  the  lest  mentioned  case  there  Is  not  todsy,  1  under¬ 
stand,  comp  Late  agreement  among  expert!  on  the  matter. 

Indeed,  aa  Is  evident  from  the  foregoing,  the  "true  Value"  of  the  magni¬ 
tude  uf  a  particular  quantity  is  Intimately  Linked  to  the  purposes  for  which  a 
value  of  the  magnitude  of  this  quantity  is  needed,  and  Its  "true  value"  cannot, 
In  the  final  analysis,  bo  dofuiaU  meaningfully  and  usefully  In  isolation  from 
these  needs.  Thnefore,  ac  this  fact  becomes  more  widely  recognised  In 
sclent.*,  and  ongineorlng,  1  hope  that  the  traditional  ttirm  "true  value"  will 
be  discarded  In  measurement  theory  and  practice,  and  replaced  by  some  inou. 
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approbate  term  suoh  as  "target  value"6  that  conveys  the  Idea  of  being  the 
value  that  one  would  Ilka  to  obtain  lor  the  purpose  In  hand,  without  any  im¬ 
plication  that  it  is  some  sort  of  permanent  oonstant  preexisting  and  transcend¬ 
ing  any  use  that  we  may  have  for  It.  I  have  retained  the  traditional  expres¬ 
sion  "true  value"  In  the  sequel  beoauee  of  Its  greater  familiarity  but  shall 
always  mean  by  It  the  relevant  "target  value." 

d.  Concepts  of  the  Precision  and  Accuracy  of  a 
Measurement  Prooess 

By  the  nreolslon  of  a  measurement  process  we  mean  the  degree  of  mutual 
agreement  characteristic  of  independent  measurements  of  a  single  quantity 
yielded  by  repeated  applications  of  the  prooess  under  specified  conditions) 
and  by  Its  accuracy  the  degree  of  agreement  of  suoh  measurements  with  the 
true  value  of  the  magnitude  of  the  quantity  concerned.  In  other  words,  the 
aocuraov  of  a  measurement  process  refers  to,  and  Is  determined  ty  the  degree 
of  conformity  to  the  truth  that  Is  characteristic  of  Independent  measurements 
of  a  single  quantity  produced  (or  producible)  by  the  repeated  applications 
of  the  process  under  specified  conditions;  whereas  Its  precision  refere  solely 
to,  and  la  determined  solely  by  the  degree  of  oonformltv  to  each  other  char¬ 
acter*  etto  of  suoh  measurements,  irrespective  of  whether  they  tend  to  be 
close  or  far  from  th««  truth.  Thus,  annnraovhas  to  do  with  closeness  to  the 


6"Wa  admit  the  existence  of  systematic  error--of  a  difference  between  the 
quantity  measured  (the  measured  quantity)  and  the  quantity  of  Interest  (the 
tr-get  quantity).  We  aak  the  observationa  about  the  measured  quality.  We 
-ibK  our  subject  matter  knowledge,  Intuition,  and  general  information  about 
the  relation  between  the  measured  quality  and  the  target  quantity."  |Ooohran, 
at  cl.  1954,  p.  33.]  1 

". .. .  Some  poople  prefer  the  term  'true  value',  although  others  excoriate 
It  4S  philosophically  unsound, 

"We  could  also  call  tha  reference  level  a  'target  value'.  In  a  way  vlile 
ir  a  bad  term  because  It  impllee  that  it  Is  something  we  want  to  find  through 
the  measurement  prooess  rather  then  something  we  ought  to  find  beoeuse, 
like  Mt,  Everest,  It  is  there.  Unfortunately  our  desires  oen  Influence  our 
notion  of  what  is  true,  and  wa  oan  even  unconsciously  bring  the  latter  Into 
agreement  with  tha  former;  my  use  of  the  term  'target  value'  Is  not  meant  to 
Imply  that  L  think  It  legitimate  to  equata  what  wa  would  like  to  saa  with  what 
Is  there."  ^Murphy  1961,  p.  265. J 
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tjru'.h,  tr^olaiaOj  only  with  olosansss  together. 

This  distinction  between  the  meanings  of  the  terms  "accuracy"  and 
precision"  as  applied  to  measurement  processes  and  measuring  Instru¬ 
ments  is  consistent  with  the  etymological  roots  of  these  words.  "Etymo¬ 
logically  the  term  'aoourate'  has  a  Latin  origin  meaning  'to  take  pains  with' 
and  refers  to  the  earn  bestowed  upon  a  human  effort  to  make  such  effort  what 
it  ought  to  be,  and  eucuracy'  In  common  dictionary  parlanoa  implies  free¬ 
dom  from  mistakes  or  exact  conformity  to  truth .  'Precise, 1  on  the  other 
hand,  hae  its  origin  in  a  term  meaning  'cutoff,  brief,  conolee';  and  'precision' 
is  supposed  to  Imply  the  property  of  determinate  limitation!  or  being  exactly 
and  sharply  defined. "  [shewhart  1939,  p.  124.1  Thu*  one  can  properly  apeak 
of  a  national,  etate,  orHooal  law  as  being  "preolee, "  but  not  ae  balng 
"acourate"--to  what  truth  oan  It  conform?  On  the  other  hand,  If  one  spoke 
of  a  particular  translation  as  being  "accurate"  this  would  imply  a  high  de¬ 
gree  of  fidelity  to  the  original  "attained  by  the  exercise  of  care,"  Whereas, 
to  speak  of  It  ai  balng  "precise, "  would  imply  myely  that  It  la  unambiguous, 
without  Indicating  whether  It  la  or  Is  not  correct. 

In  spite  of  the  dletmot  difference  between  the  etymological  meanings 
of  the  terms  "accuracy"  end  "precision,"  they  araireatedea  synonyms  In 
m.'-iy  standard  dictionaries;  and  Merrlam- Webster [1942],  after  drawing  the 
helpful  distinctions  quoted  In  the  foregoing  footnote,  promptly  topples  the 
:,iructure  so  carefully  built  by  adding  " scrupulous  exactness"  as  an  alter¬ 
native  meaning  of  "precise."  Consequently  it  is  not  surprising  that  "There 
ate  probably  few  words  as  loosely  uied  by  scientists  as  precision  and 
accuaov  .--It  la  not  unusual  to  find  them  uned  Interohangnably  In  scientific 
writ! nge."  ^Sohrock  1950,  p.  io,j 


7It  is:  sometimes  helpful  to  dietlnguleh  between  “correct, "  "aoourate,"  ana 
"exxi  i";  "CORRECT,  the  moat  colorless  term,  Implies  scarcely  more  than 
freedom  from  the  fault  or  error,  as  Judged  by  some  (usually)  conventional  or 
acknowledged  standard;. . .  ACCURATE  Implies,  more  positively,  fidelity  to 
loot  or  truth  attained  by  the  exerolsa  of  oars; .  • .  EXACT  smphaeliea  the 
Btrlotnesa  or  rigor  of  the  agreement,  which  neither  cxoceda  nor  falls  onou 
of  the  fact,  standard  or  truth:, . ,  PRECISE  stresses  rather  sharpness  of  de¬ 
finition  or  delimitation. . ."  | Merrlam- Webster  1942  p.  2031  . 
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On  the  other  hand,  aa  Shewhart  haa  remarked: 


"Careful  wrltors  tn  the  theory  of  errora,  of  courae,  have  always  inalated 
that  accuracy  Involves  m  tome  way  or  other  the  difference  between  what  La 
obeerved  and  what  tu  true,  whereaa  precialon  involvea  the  concept  of  repro¬ 
ducibility  of  what  la  cbaerved.  Thua  Laws,  writing  on  electrical  meanure- 
ments,  eay»:8  'Every  experimenter  muat  form  hla  own  eetlmate  of  the  aooureoy, 
or  approach  to  the  absolute  truth  obtained  by  the  uae  of  hla  Inatrumenta  and 
proceaaes  of  maaaurement.  He  muat  remember  that  a  high  precialon,  or 
agreement  of  u, o  rmulta  among  themaelvea,  la  no  Indication  that  the  quan- 
tity  under  measurement  has  been  accurately  determined.'  As  another  ex¬ 
ample  we  may  take  the  following  comment  from  a  recent  and  authorLtatlve 
treatise  on  chemical  analysis:9  'The  analyst  enould  form  the  habit  of  esti¬ 
mating  the  probable  acouraoy  of  his  work.  It  is  a  common  mletake  to  confuse 
aacuracy  and  preoiaton.  Accuracy  is  a  measure  of  the  degree  of  correctness. 
Precision  la  a  measure  of  reproducibility  in  the  hands  of  a  given  operator.'" 
[Shewhart  1939,  pp.  124-125.] 


More  recently,  Lundall,  Hoffman,  and  thalr  associates  at  the  National  Bureau 
of  Standards  have  re-emphasised  the  importanoe  of  the  distinction  between 
"praolelon"  and  "accuracy": 


"In  discussions  of  chemical  analysis,  the  terms  precision  and  accuracy 
""o  often  used  Interchangeably  and  therefore  Lnoorreotly,  for  precision  Is  a 
measure  of  reproducibility,  whereas  acauracy  Is  a  measure  of  correotn***. 
The  analyst  Is  vitally  interested  In  both,  for  his  results  mu«t  be  sufficiently 
accurate  for  the  purpose  In  mind,  and  he  cannot  aohieve  aoouraoy  without 
piuolsion,  especially  sines.  Ms  reported  result  is  often  bo?., -a  on  one  deter¬ 
mination  ami  lately  on  more  tl.uii  three  determinations.  The  recipient  of  the 
analysis  is  Interested  in  aoouraoy  alone,  and  only  In  accuracy  sufficient  for 
hi «  purposes. "  [HlllebranU  et  al, ,  1953,  p.  3,1 


*VrankA.  Laws,  Electrical  Measurements,  p.  593  iMouraw  Hill,  New  York, 
N  .  Y. ,  1917). 

90,  E.  E,  Lundell  and  J.  I,  Hoffman,  Outlines  of  Methods  of  Ohemloal 
Analysis,  p,  220  (John  Wiley  and  Sons,  New  York,  N.  Y. ,  1938). 
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It  lb  most  unfortunate  that  In  everyday  parlance  we  often  speak  of 
"accuracy  and  precision, "  because  accuracy,  requires  dfiEl.liilD  but  JH.slatOn. 
does  not  necessarily  imply  accuracy  ■ 

"It  is,  in  fact,  interesting  to  compare  the  measurement  situation  with 
that  of  a  marksman  aiming  ac  a  target.  We  would  call  him  a  precise  marks¬ 
man  if,  In  itring  a  sequenue  of  rounds,  he  were  able  to  place  all  his  shots 
in  a  rather  small  circle  ori  tn«  target.  Any  other  rifleman  unable  to  group  his 
shots  in  such  a  small  circle  would  naturally  be  regarded  as  less  precise. 

Mu:,l  poople  would  accept  this  characterisation  wnether  either  rifleman  hits 
the  bull eye  or  not. 

"Surely  all  would  agree  that  if  our  man  hits  or  nearly  hits  the  bull's- 
eye  on  all  occasions,  he  should  bs  called  an  accurate  marksman.  Unhappily, 
he  may  be  a  very  precise  marksman,  but  If  his  rifle  Is  out  of  adjustment,  per¬ 
haps  the  small  circle  of  shots  Is  centersd  at  a  point  some  distance  from  the 
bull's-eye,  In  that  case  we  might  regard  him  as  an  inaccurate  marksman. 
Perhaps  we  should  say  that  ha  Is  a  potentially  accurate  marksman  firing  with 
a  faulty  rifle,  hut  speaking, categorically,  we  shquld  have  to  say  that  the 
results  were  Inaoourate."  (  Murphy  1961,  p.  265. j 

It  follows  frum  what  has  been  said  thus  far  that  "If  the  precislone  of 
two  (Mooesses  are  the  same  but  the  btasee  are  different ,  the  prooeos  of 
Pina  Her  blao  may  be  aatd  to  have  higher  accuracy  while  If  the  biases  ere 
both  negligible,  the  prooees  of  higher  precision  may  be  said  to  have  higher 
nccu’vjay."  Unfortunately,  "In  other  caiaa  such  a  simple  comparison  may 
bo  lm'.fk?afllbto * “  [AI3TM  1961#  p*  1760.1 

To  fully  apr^olate  the  preceding  statement— and  especially  the  diffi¬ 
culty  of  comparing  accuracies  In  some  u«*sas— let  us  consider  figures  l  and 
2,  in  wnlcli  the  origins  of  the  soales  correspond  to  the  true  value  of  T.w.1 
the  quantity  measured,  so  that  the  curves  shown  may  be  regarded  ae  depleting 
the  distributions  of  errors  of  the  measurements  yielded  by  a  selection  01 
different  measurement  prooeesee.  Consider  first  the  three  symmetrical  dis¬ 
tributions  m  tho  top  half  of  figure  1.  All  three  of  these  distributions  ere 
centered  on  sero,  so  that  these  measurement  processus  havo  no  bias  .  it  le 
uvldont  Hull  the  process  ofhlghest  precision,  a  ,  Is  alec  the  process  of 
highest  accuracyi  and  that  the  piooeee  of  least  preotslon,  a,  Is  also  the 
process  of  least  accuracy.  Sine#  curve  b  In  the  upper  half  of  figure  1  and 
curve  d  In  the  lower  half  have  Identical  eUe  and  shape,  thn  corresponding 
processes  have  the  same  precision,1  but  process  b  Is  without  bias,  whereas 
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process  U  has  a  positive  bias  of  two  units,  so  that  prooess  b  Is  clearly  the 
more  aoBurate .  (In  particular  we  may  note  that  whereas  It  Is  practically 
certain  that  process  b  will  not  yield  a  measurement  deviating  from  the  truth 
by  more  than  two  units,  exactly  one-half  of  the  measurements  yielded  by 
prooess  d  will  deviate  from  the  truth  by  this  much  or  more.)  Similar  remarks 
clearly  apply  to  processea  o  and  e  corresponding  to  curve  c  Ln  the  upper 
half  and  curve  e  In  the  lower  half  of  figure  I,  but  ln  this  Instance  the  supe¬ 
riority  of  process  c  relative  to  process  e  with  respeot  to  accuracy  Is  even 
more  marked.  (In  particular,  we  may  note  that  whereas  It  Is  practloelly 
certain  that  no  measurement  yielded  by  process  c  will  devlste  from  the  truth 
by  as  much  as  one  unit,  It  is  practically  certain  that  every  measurement 
yielded  by  process  o  will  deviate  from  the  truth  by  more  than  one  unit,) 

figure  2,  which  Is  eeeent.tally  the  same  as  one  given  by  General 
Simon  194b,  fig.  1] ,  portrays  three  meaeurement  prooeases  A,  B,  and  0, 
differing  from  each  other  with  respect  to  both  precision  end  bias.  Comparison 
of  these  three  prooaeuee  with  respeot  to  aoouraov  Is  not  quite  so  simple, 

First,  it  is  evident  that,  although  process  A  has  greater  precision  than  pro¬ 
cess  B,  prooeas  B  Is  the  more  accurate  of  the  two.  (In  particular,  It  is 
practically  certain  that  none  ot  the  measurements  yielded  by  prooess  Jj  will 
deviate  from  the  truth  by  morn  than  4  units,  whereas  SO  parcent  of  the  meas¬ 
urements  irom  prooess  A  will  deviate  from  Iho  truth  by  four  unite  or  more.) 
Noxt,  is  process  B  more  (or  lese)  aoourate  than  process  C  which  is  unbiased. 
out  has  a  very  low  precision?  Prooess  B  has  a  positive  bias  of  t.wo  unite, 
but  has  sufficiently  greator  precision  than  process  C  to  also  have  greater 
aopi”T.ov  than  prooeas  0  .  (While  approximately  SO  percent  of  tho  measure- 
mont,.  yielded  by  process  0  will  deviate  from  the  truth  by  more  than  two 
unite  (In  either  direction),  and  M.rr.otly  SO  percent  of  the  measure, enls  yielded 
by  process  B  wll<  deviate  from  ilia  truth  by  two  units  or  more  (In  tho  positive 
direction  only),  It  cannot  be  Ignored  that  about  10  parcert.  of  tho  measure¬ 
ment*  yielded  by  prooeas  G  will  deviate  from  the  truth  by  four  unit,,  c 
more  whereas  it  Is  practically  oortaln  that  no  measurement  yielded  by  bro- 
cjbds  B  will  deviate  from  the  truth  by  as  much  as  four  units.)  Similarly,  it 
may  bo  argued  that  process  A,  In  spite  of  Ite  bias,  has  greater  aunurnov 
th»iii  process  0  "since  the  range  In  measuremenlc  of  0  more  than  covers 
the  corresponding  ranges  ot  A  or  B."  [Simon  1946,  p,  654.1  While  thl* 
conclusion  that  of  the  throe  mensurement  processes  daplotid  in  figures  2, 
ptooeis  G  has  the  least  accuracy  ,  may  not  be  anttraly  acceptable  to  some 
persons,  It  la  consistent  with  Gauss'  dictum,  in  a  latter  to  F,  W,  Bessel, 
to  the  affect  that  maximlatng  the  probability  of  a  sera  error  Is  lass  Important 
than  minimising  the  "averinje"  Injurious  effects  of  errors  In  general.  [O,  F. 
Gauss,  1839,  pp.  146-147.J  1 
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Before  leaving  figure  2,  we  must  not  fall  to  join  General  Simon  in  re¬ 
marking  that  "the  average  of  a  large  number  of  measurements  from  |  process! 

C  will  be  more  accurate  than  a  similar  average  from  either  A  or  B"  u  Simon 
1946,  p.  654 J  .  This  point  is  actually  Illustrated  in  our  figure  1:  the  three 
curves  in  theJtop  half  of  figure  1  portray  the  distributions  of  errors  of  single 
mee^surements  (curve  a)  of  averages  of  12  measurements  (cuive  b)  and 
averages  of  144  measurements  (curve  c)  from  process  C  ;  and  curves  d  and 
e  in  the  lower  half  show  the  distributions  of  errors  of  individual  measure¬ 
ments  (curve  d),  and  of  averages  of  12  measurements  (curve  e)  from  process 
B,  respectively.  It  Is  evident  that  averages  of  12  measurements  from  process 
C  (curve  b  in  upper  portion  of  fig.  i)  have  not  only  greater  accuracy  than 
Individual  measurements  from  process  B  (curve  d  in  lower  portion  of  the  figure), 
but  also  greater  accuracy  than  averages  of  12  measurements  from  process  B 
(curve  e  in  lower  portion).  • 


On  the  other  hand,  it  is  obvious  that,  if  our  choice  is  between  indivi¬ 
dual  measurements  from  process  C  (curve  a)  and  averages  of  12  measurements 
from  process  B  (curve  e),  the  latter  will  clearly  provide  greater  accuracy  . 

In  brief,  a  procedure  with  a  small  bias  and  a  high  precision  can  be  more 
accurate  than  an  unbiased  procedure  of  low  precision.  It  is  important  to 
realize  this,  for  In  practical  life  it  Is  often  far  better  to  always  be  quite  close 
to  the  true  value  than  to  deviate  all  over  the  place  in  individual  cases.. but 
strictly  correct  "on  the  average, "  like  the  duck  hunter  who  put  one  swarm 
of  shot  ahead  of  the  duck,  and  one  swarm  behind,  lost  his  quarry,  but  had 
the  dubious  satisfaction  of  knowing  that  in  theory  he  had  hit  it  "on  the  average. 
'T'Ms  we  must  remember:  In  practical  life  we  rarely  make  a  very  large  number 
■  i  measurements  of  a  given  type — we  can’t  wait  to  be  right  on  the  average — 
our  measurements  must  stand  up  in  individual  cases  as  witon  as  possible. 


Despite  the  foregoing,  freedom  from  bias,  that  is,  freedom  from  "large” 
bias,  is  a  desirable  characteristic  of  a  measurement  process.  After  all  v= 
■v::nt  our  measurements  to  yield  us  a  determination  that  we  can  use  as  a 
substitute  for  the  unknown  value  of  a  particular  magnitude  whose  value  wo 
need  for  some  purpose--we  don’t  want  a  determination  of  the  value  of  some 
other  magnitude  whose  relation  to  the  one  we  need  is  indefinitely  known. 

In  view  of  the  difficulty  of  comparing  with  respect  to  accuracy  meas¬ 
urement  processes  that  differ  both  in  bias  and  precision,  some  writers  have 
elected  to  take  the  easy  way  out  by  defining  "accuracy"  to  be  equivalent  to 
absence  of  bias,  saying  that  two  measurement  processes  having  different 
biases,  the  process  of  smaller  bias  is  the  more  "accurate”  regardless  of  the 
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relation  of  their  respective  proclatone.  (Sat,  for  example,  Bear ■  I  19 53 ,  p.  4], 
Oetle[l854,  p.  4j,  and  Schenok  |1B61,  p.  4,  p.  14j.)  Whlla  tha  adoption  J 
of  thtroonoept  of  ^acouraoy"  certainly  makea  the  discussion  of  "acouraoy" 
and  "precision"  simpler  for  the  authors  concerned,  this  practice  is  contrary 
to  the  principle  of  "conservation  of  linguistic  resources,"  as  R.  B.  Murphy 
puts  it,  addlngi  "It  seems  to  mo  that  the  terma  'bias'  and  'systematic  error' 
are  adequate  to  oover  the  situation  with  which  they  are  concerned.  If, 
nevertheless,  we  add  the  term  'accuracy1  to  apply  again  In  this  restricted 
sense,  we  are  left  wordlast--at  the  moment  at  laeet--when  it  comen  to  the 
idea  of  ovar-al!  error.  From  tho  point  of  view  of  tho  need  for  a  term  It  Is  hard 
to  defend  the  view  that  accuracy  should  concern  Itself  solely  with  bias, , , 
[and]  there  Is  overwhelming  evidence  that  we  nood  a  term  at  leaat  for  the 
concept  of  over-all  error."  [Murphy  1861,  pp.  265-266,1 

3.4.  Mathematical  Specification  of  the  Picolsl on  of  a  Measurement 
Process. 

a.  Simple  Statistical  Control 

Let  ua  now  aonslder  the  mathematical  definition  of  the  preolslon  of  a 
measurement  process  under  a  fixad  sat  of  circumstances.  By  definition, 

Uih  pi'onlslon  of  a  measurement  process  has  to  do  with  the  "oloeenesi  to- 
Tothur"  that  la  typical  of  successive  meeeuiements  of  a  single  quantity  gen- 
nrated  by  applications  of  the  precast  under  these  fixed  conditions.  Other¬ 
wise  expressed,  It  has  to  do  with  the  typical  "closeness  togethpi"  of  the 
two  individual  measurement!  constituting  an  arbitrary  pair,  If  the  expres¬ 
sion  "‘vploal  'closeness  together'"  Is  to  be  meaningful,  the.  measure  manta 
generated  by  repeated  application  nf  tha  process  to  tne  measurement  of  a 
single  quantity  must  be  homogenrinus  In  soma  sense.  Therefore,  for  the 
moment.  let  us  assume  that  the  measurement  process  la  In  a  stale  of  sliaatl 
statistical  control,  so  that  the  suoaeaslvo  measurements  in  each  of  the 
sequeoooa  (1),  (l  *  l,  2,  3,.,,),  generated  by  the  process  may  &\\_  be  re¬ 
garded  as  "observed"  values  of  Independent  Identically  distributed  random 
variables. 

Just  as  we  may  regard  each  individual  measurement  In  a  particular 
sequence  (1)  aa  striving  to  oxprcsi  the  value  of  the  limiting  mean  a  ,  so 
also  we  may  regard  each  Individual  dlfferenae  xy  -  x^y,  ,  1  ^  k,  as  striving 

to  express  the  characteristic  spread  between  an  arbitral*/  pair  uf  measure¬ 
ments,  x'  and  x",  say.  For  this  purpose  the  signs  of  these  differences  are 
clearly  irrelevant.  Therefore,  by  analogy  with  our  use  of  a  sequence  of 
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cumulative  arithmetic  means,  (2),  to  achieve  a  mathematical  formulation  of 
tne  concept  of  a  limiting  mean  associated  with  measurement  of  a  given  quan¬ 
tity  by  a  particular  measurement  process,  let  us  adopt  the  sequence  of  cumu¬ 
lative  arithmetic  means  of  the  aam»£jUL  of  the  Jh(il  '  0/2  distinct  differences 
among  the  first  u  measurements  of  a  particular  sequence  (1),  for  example, 
the  sequence 


(3) 


2 _ 

n(n  -  1) 


( 


'Ik 


)7  , 


(n  -  2,  3 , 


as  the  basis  of  a  matuematical  formulation  of  the  concept  of  the  precision 
of  a  measurement  process. 


The  necessary  and  sufficient  condition  for  almost  sure  convergence  of 
the  sequence  (3)  o  a  finite  limit,  say  Is  that  the  Strong  Law  of  Large 
Numbers  be  applicable  to  the  sequence. 


<4) 


*ir 


•  »  t 


consisting  of  the  squares  cf  the  corresponding  terms  of  the  original  sequence 
'1 ) .  (Boundedness  of  the  x’s  In  addition  to  statistical  control  is,  lor  example, 
Mifflclent  to  ensure  that  the  sequence  (4)  will  also  obey  the  Strong  Law  oi 
Large  Numbers.)  H  the  Strong  Law  of  Lorge  Numbers  is  applicable  to  the 
sequence  of  squares  (4),  and  If  the  measurement  process  Is  In  a  stats  of 
simple  statistical  control,  then  the  cumulative  arithmetic  mesne  of  the  squares 
of  the  measurements,  thrt  Is,  the  sequence 


(5) 


n 


I 

J  -  1 


xj/n, 


(n  -  1,  2,  ,.), 


will  almost  surely  tend  to  a  limit,  say  S,  the  magnitude  ot  which  will  depend 
on  the  quantity  measured,  the  measurement  process  involved,  but  not  on  the 
"occasion"  (identified  by  the  subscript  "1").  By  virtue  of  an  algebraic  Identity 
that  Is  well  known  to  students  of  mathematical  inequalities,  namely, 


* 
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j  •  1  1  1  j  «  1  2  j  «  j  k  -  1  J  * 


(n  2  2) 


and  of  the  feat  that  the  right-hand  side  of  (6)  Is  always  positive  except 
when  the  a's  are  al!  equal,  it  la  easily  seen,  on  dividing  both  sides  of  (6) 
by  n  ,  that  S  will  always  exceed  /J.2,  the  square  of  the  (almost  sure)  limit 
of  the  sequence  (2),  so  that  we  may  write  S  ■  jj.2  +  O'2,  with  cf2>  o. 
furthermore,  applying  the  algebraic  Identity  (6)  In  reverse  to  the  right-hand 
side  of  (3)  yields  the  following  relationship  between  the  corresponding  terms 
of  sequences  (3),  (5),  and  (l): 


)ln-(xin)2  >  0,  In  2  2)* 


«*>,„  -  2 


Hence,  If  a  measurement  process  Is  In  a  state  of  simple  statistical  control 
and  the  Strong  Law  of  Large  Numbers  Is  applicable  to  a  sequence  of  squared 
mvar.urements  (4),  then  the  sequence  (d2)ln,  deflnectby  (3),  will,  In  view  ot 

(7) ,  tend  almost  surely  to  a  finite  limit  A2  -  2  0TZ,  Thus  we  see  that  O' 2 , 
termed  the  variance  of  the  measurement  process,  Is  the  m>.un  value  of  one- 
half  of  the  squared  difference  between  two  arbitrary  maaevromonts  x’  and  i»", 
thru  is, 

(8)  n 2  -  i  w  ••  x")* 

and  uiovldes  an  Indication  of  the  imprecision  of  the  proeeos.  The  square 
root  of  the  variance,  O’,  Is  termed  the  standard  deviation  of  the  process. 

It  Is  natural,  therefore,  on  the  basis  of  a  single  sequence  at  n  mi-so- 
surements  of  a  single  quantity,  to  take 


92  i  i(?) 


,  n-1  n  „ 

^rnr(F,  V— ^.r — 
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ai  the  sample  estimate  of  the  underlying  variance  O'2;  and  the  square  root, 
as  the  sample  estimate  of  O'.*® 

From  (9),  since  Z  a  xn  tends  (almost  oureiy)  to  fx  it  la  evident  that 
O'2  Is  also  the  mean  value  of  the  squared  deviations  of  Individual  maasurt- 
monts  from  the  limiting  mean  fJ.  of  the  process,  that  Is  Cf 2  *  (x  -./x)2i  so 
that  the  standard  deviation  O'  may  be  regarded,  In  the  language  of  meohanlas, 
an  the  radius  of  gyration  of  the  distribution  of  all  possible  measurements  a 
about  jj.  ,  the  limiting  mean  of  tha  process. 


Remark;  Mathematically  the  foregoing  dlecusiton  can  be  carried  out 
equally  well  In  terms  of  the  absolute  (unsigned)  values  of  tha  differences 
instead  of  in  terms  of  their  equarea.  3uoh  an  approach  le,  mathematically 
speaking,  somewhat  more  general  In  that  It  requires  for  Its  validity  merely 


that  tha  Strpng  Law, of  Large  Numbers  be  applicable  to  the  sequenae 

lxLll  >  Ixl2l  <  '"•  i*J  i  •  •  •  of  absolute  valuaa  of  tho  xtj  rather  then  to  the 

sequence  (4)  of  their‘»quares ,  From  the  practical  viewpoint,  however,  this 


greater  generality  is  entirely  Illusory,  and  tn#  mathematics  of  absolute  values 


of  varlablss  Is  always  mors  cumbersome  than  the  mathematics  of  their  squares. 
For  example,  tha  arithmetic  mean  of  the  abiolute  valuee  of  the  n(n  -  l)/2 


distinct  differences  among  jq,  measurements,  l.e., 


(10) 


n  -  1  n 

— * —  E  E 

n(n  -  1)  j  »  i  k  «  H 


is  jist_  expressible  ai  a  multiple  of  the  sum  cf  the  aosolute  deviations  of  the 
measurements  from  their  menu  -  y  |  >  r<nd  for  large  valuee  of  A  the 


>(jFtcm  the  algebraic  Identity  (5),  it  Is  evident  that  tha  practice  In  some  circle*, 
of  dividing  £  (x  -  I!)2  By  £,  Inetead  of  n  -  1,  amounts  to  Inoludlrg  uach 

J  ■  1  , 

of  the  distinct  squared  differences  (xj  -  x^)2  ,  J  ^  k,  twice  In  the  summation, 
togethsr  with  £  Identically  sera  terms  (xj  -  x^)z,  J  -  k,  each  included  once, 
and  then  dividing  by  n2,  the  total  number  of  term*  (real  and  phantom)  Involved. 
Viewed  In  this  light  It  would  seem  that  dtvlelon  by  n  -  l  Is  more  reasonable, 

In  that  the  Inclusion  of  Identically  zero  terms  In  the  formuletlon  of  a  measure 
of  variation  Is  a  bit  unreasonable . 
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evaluation  ot  (10)  praaanta  computational  difficulties.  The  approach  in  forme 
of  the  abaoluta  valuaa  ot  the  dtffsrenoes  alao  has  the  disadvantage  from  the 
practical  viewpoint  that,  a  a  we  shall  see  In  a  moment,  component  a  of  imare- 
cl.al.on  are  additive  In  terms  of  squared  quantities  such  as  or  *,  so  that  In 
thia  sense  the  variance  g  Ms  a  more  appropriate  measure  of  the  dtaperalon 
of  the  x'«  about  their  limiting  mean  fx  than  la  (Titself. 

Ordinarily,  the  magnitude  of  O' 2  (and,  hence,  of  0*  ),  unlike  that  of 
/u.,  depends  only  on  the  measurement  process  concerned  and  the  clroum- 
stancea  under  which  it  is  applied,  and  not  also  on  the  magnitude  of  the 
quantity  measured- -otherwise  we  oould  not  speak  of  a  measurement  process 
having  a  variance,  or  a  standard  deviation. 

Since  the  preolelon  nf  the  procnee  obviously  decreases  as  the  valuu 
of  cr  (or,  of  c*)  increases,  and  vice  versa,  It  le  necessary  to  take  some 
inverse  function  of  (rtea  measure  of  the  precision  of  process.  To  conform 
with  traditional  usage  It  Is  neoassary  to  regard  the  preolelon  of  e  meaaure- 
ment  procab.  as  Inversely  proportional  to  Its  standard  deviation  O' which 
Is,  therefore,  a  measure  of  the  Imprecision  of  the  process.  Thus,  Clause, 
writing  In  1809,  remarked  that  hie  oonstent  h  ■  l/CT'yloould  properly  be  con¬ 
sidered  to  be  a  measure  of  the  preolelon  of  the  observations  because  If,  for 
exempio  h‘  -  2h,  that  la,  If  o'1  ■  1/2  0*  ,  then  “a  double  error  oan  be  oom- 
miuud  in  the  former  system  with  the  eama  facility  as  a  single  error  In  the 
'.itt.vi ,  In  which  case,  according  to  the  common  way  of  speaking,  a  double 
degree  of  precision  is  attributed  to  the  latter  observations."11 

'"he  fact  of  the  matter  la,  however,  than 

"...  different  fields  have  particularly  favorite  ways  of  expressing 
precision.  Moat  of  these  measure*  arc  multiples  of  the  standard  deviation; 

It  Is  nut  alwayu  clear  which  multiple  le  meant. . , . 


"Ceterum  oonstans  Jb,  tomquam  mensure  praeclslunls  obner vatlonum  con- 
lilderml  poterit.  ...  Quodst  igitur  e.g.,  h'  “  kh,,  aeque  faolle  in  eyst^ir.'.ta 
priori  error  duplex  oommittl  potent;  ao  simplex  In  posteriori,  in  quo  casu 
observation!  thus  pgsterioribus  secundum  vuigsrem  inquendl  morem  praeolslo 
duplex  tribultur."  [Gauss  180,9,  .Art.  178;  1671,  p.  233;  English  translation, 
1857,  pp.  ZbS-ZbO.1] 
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"Some  consider  it  unfortunate  that  precision  should  tea  stated  as  a 
multiple  of  standard  deviation,  since  precision  should  increase  as  standard 
deviation  decree ses,  Indeed,  It  would  be  more  exact  to  say  that  standard 
deviation  is  a  measure  of  Imprecision,  However,  sensitivity,  as  ws  have 
previously  Indicated,  suffers  nom  this  logical  Inversion  without  hurt,  Per¬ 
haps  we  can  best  avoid  this  by  saying  that  standard  deviation  Is  an  Index 
of  precision.  The  habit  of  saying  'The  precision  Is  . . Is  deeply  rooted, 
and  there  would  be  understandable  impatlenoe  with  the  notion  that  standard 
deviation  should  be  numerioally  Inverted  before  being  quoted  In  a  statement 
of  precision."  ^Murphy  1961,  pp.  266-267. j 

In  consequence  the  ASTM  has,  at  loaat  tentatively,  taken  the  following  posi¬ 
tion! 


"The  numorloal  value  of  any  commonly  uasd  Index  of  preotslon  will 
be  smaller  the  more  closely  bunched  are  the  Individual  measurements  of  a 
process.  As  more  causes  are  added  to  the  system,  the  greater  the  numeioal 
value  of  the  Index  of  precision  will  ordinarily  become.  If  the  same  Index  of 
precision  is  used  on  two  different  processes  based  on  the  same  method  or 
intended  to  measure  ths  same  physioal  pioperty,  the  prooess  that  has  the 
smaller  value  of  the  index  of  precision  is  said  to  have  higher  precision. 

Thus,  although  the  more  usual  Indexes  of  precision  are  really  direct  msasuiea 
of  imprecision,  this  inversion  of  reference  has  been  firmly  established  by 
custom.  The  value  of  the  seleoted  Index  of  precision  of  a  prooess  Is  referred 
to  simply  as  Its  precision  or  Its  stated  precision."  ^ASTM  1961,  p.  1759. j 

As  ws  have  remarked  previously,  In  practical  work  ths  end  result  of 
measuring  some  quantity  or  calibrating  an  Instrument  for  a  standard  rarely 
consists  of  d  single  measurement  of  the  quantity  of  ln^orvMt.  More  often 
It  Is  some  Kind  of  average  ut  adjusted  value,  for  example,  the  arithmetic 
mean  of  a  number  of  Independent  measurements  of  the  quantity  of  Interest, 

U't  ue,  therefore,  consider  the  statistical  properties  of  a  sequence  ot 
itivtlc  means  of  successive  nonoverlapping  groups  of  n  measurements  each 
from  a  sequence  (1)  of  individual  measurements  yielded  by  a  measurement 
P'.ooeis  on  a  particular  occasion.  In  other  words,  let  us  consider  the  se¬ 
quence 

(11)  *t  1  >  *1 2 1  •  •  •  *  m '  ■ 1  ’ 


of  distinct  arithmetic  means  of  n  measurements  each 
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02)  «im  -  n  £  x  -  (m-  l. 

]  ■  (m-l)n-t-l  1 


derived  from  a  sequence  (1)  of  individual  meaeurements  of  a  single  quantity 
produced,  or  at  laait  conceptually  producible,  by  the  meaiuremeut  prooeaa 
concerned  on,  eay,  the  ith  occaalon.  If  the  "underlying  measurement  ptooess" 
giving  rue  to  the  individual  measurements  la  in  a  state  of  simple  statistical 
control,  then  the  "oxtended  measurement  prooess"  giving  rise  to  the  averages 
xLm  will  also  be  Lr.  a  state  of  simple  statistical  control,  Consequently,  the 
mathematical  analysts  of  section  3.2,  but  with  the  averages  xlm  In  place  of 
the  individual  measurements  Xy,  will  oarry  through  without  other  change. 

Let  p.  -  denote  the  limiting  mean  thus  associated  with  the  "extended  meas¬ 
urement  process"  giving  rise  to  the  averages  Rtmas  Its  "Individual"  meas¬ 
urements.  Since  the  cumulative  arithmetic  mean  of  the  first  m  terms  of  the 
sequence  (11)  Is  the  same  as  the  cumulative  arithmetic  mean  of  the  first 
mn  terms  of  the  sequenoi  (1)  of  Individual  measurements,  It  is  clear  that 
the  Limiting  mean/'<g  associated  with  the  sequenoe  of  averages  (11)  Is  the 
name  as  the  limiting  mean  essooUted  with  the  original  sequence  (1)  of  In¬ 
dividual  measurements,  that  is, 

(13)  fXK  »fl 

similarly,  the  mathematical  analysis  at  the  beginning  of  the  present 
section,  but  wlth_the  Individual  measurements  in  (3)  thtu  (9),  replaced 
by  the  nvarages  x^,  carries  through  essentially  as  before.  Let  0  2  de¬ 
note  thu  variance  thus  associated  with  the  "extended  measurement  process" 
giving  rise  to  lh«  ■tmusuue  of  avenges  (11).  As  In  the  case  of  the  varlenoe 
O'2  of  Individual  measurements,  rs  also  may  (Y2  be  Intarprated  as  the 
overall  mean  valua  of  tha  squared  deviation  of  "Individual"  avsragoa  „ 
from  the  limiting  mean  Mr  of  the  "extended  prooeaa,"  that  la, 


(M)  ar'J  -  (x  -  "  «  —  ix  >2 


By  virtue  of  the  algebraic  Identity 
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,  T  n  1S  r  n 

(15)  (x  -  ^  )2  ■  1  £  x,  — p.  ■  1  £  (x,  -  fx  ) 

n  )  -  1  1  n  j  -  1  1 

L  J  L 

i  T  n  „  n  •  l  n 

“  “2  E  (Xj  -  fX  )2  *  2  I!  t  (x,  -  n  >  (xk  -  fx  ) 

n  J  ■  1  j  -  i  k  -  J  +-1 


It  la  raadlly  seen  that 


n  n 


(Tha  maan  valuo  of  a  aum  la  always  tho  aum  of  the  mean  valuaa  of  lla  Indivi¬ 
dual  tarma,  ao  that  tha  ovarall  maan  valua  of  the  firat  aummatlon  Inalda  tha 
brackets  In  tha  laat  Lina  of  (15)  la  almply  n  O'  ■  .  Furtharmora,  In  tha 
oaaa  of  lndapandant  idantloally  dlatrlbutad  measurements,  tna  ovarall  maan 
valua  of  tha  tarm  Involving  tha  doubla  aummatlon  la  0.) 

Slnoa,  from  (16),  0*x  -  O'/yfT  ,  it  la  a«an  that  tha  praolalMof 

tha  arithmetic  maan  of  n  lndapandant  measurements  la  proportional  toyn. 
Hanoe  tha  arithmetic  mean  of  1  independent  maaauramanta  naa  double  tha 
praolalon  of  a  single  maaauramant)  tha  maan  of  J|  Inaapatidant  maaauramanta, 
thriaa  tha  praolalon  of  a  single  maaauramant,'  and  ill  lndapandant  maa*"Vi*  ■ 
manta  will  ba  required  If  their  arithmetic  maan  la  to  have  *  12- fold  Increase 
in  preclcicn  over  a  single  measurement.  (But  to  ask  for  a  12-fold  Inoraaaa 
in  praolalon  is  to  eak  for  e  "ary  considerable  Improvement  indeed,  as  oan  ba 
naan  from  a  comparison  of  curves  a  and  o  In  the  too  half  of  fiv.  !  ) 

Tn  serve  aa  a  reminder  of  tha  distinction  between  tha  standard  rievin  • 
non  of  an  Individual  maaauramant  end  tha  standard  davlallun  of  a  mean  ft, 

It  la  ouitomery  to  refer  to  0*  as  tha  "standard  deviation"  of  a  single  meas¬ 
urement  x,  and  to  CT  R  aa  the  11  atandard  error  "  of  the  (arithmetic)  mean  ft  . 
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b.  Within- Occasion*  Oontrol 

In  tha  foregoing  It  has  bean  aiaumad  that  the  Individual  maaauramenta 
comprising  tha  aaquanoaa  (1)  corresponding  to  the  maoeotlve  "occasions," 

(1  «*  l,  2,  . ..),  could  all  ba  regarded  aa  "obaerved  value*"  of  indaoandent 
identically  distributed  random  variable*,  that  li,  that  tha  maaiurament  pro¬ 
cess  concerned  was  In  a  state  of  simple  statistical  oontrol.  Whan  suoh  is 
the  case  then  any  subsot  of  n  measurements  is  strlotly  comparable  to  any 
other  subset  of  n  measurements,  and  any  two  suoh  subsets  can  be  combined 
and  regarded  validly  as  a  single  set  of  2n  measurements.  Unfortunately, 
as  Student's  comment  quoted  on  page  484  above  clearly  Implies,  suoh  com¬ 
plete  homogeneity  of  measurement  i*  rarely  If  ever  met  in  praotloe.  More 
often  the  situation  is  as  described  by  Sir  Qaorgo  Blddoll  Airly,  British 
Astronomer  Royal  1835-1881,  In  (to  my  knowledge)  the  first,  elementary  book 
on  the  theory  of  errors  and  combination  of  observations  in  the  English  lan¬ 
guage  [Airy  1861,  p.  92  J  i 

"Whan  successive  sartai  of  observations  ora  made,  day  after  day, 
of  th*  sama  measurable  quantity,  whloh  la  either  Invariable. . .  or  admits 
of  being  reduced  by  calculation  to  an  Invariable  quantity. . , t  and  whan  every 
known  Instrumental  oorreotlon  has  bean  applisd. . ,)  still  it  will  sometimes 
be  found  thet  the  reault  obtatned  on  ona  day  differs  from  tha  result  obtained 
on  eiioiner  day  by  a  larger  quantity  than  could  have  bean  Anticipated.  The 
Idm  than  presents  itself,  that  possibly  thsra  has  bean  on  soma  ona  day,  or 
on  every  day,  some  causa,  special  to  the  day,  whloh  has  produced  n  Con¬ 
stant  Error  In  ths  measures  of  that  dsy." 

Sir  George,  howtver,  cautions  ug.-ii.ut  Jumping  to  conclusions  on  the  basis 
of  only  *  few  observational 

"The  existence  of  a  daily  conetant  error. .  .ought  not  to  ba  lightly 
assumed,  Whan  obeervettons  era  made  on  only  Iwo  or  three  days,  andtho 
number  of  observations  cn  each  day  Is  not  extremely  great,  the  mare  fact, 
of  accordance  on  each  day  end  discordance  from  day  to  day,  1*  not  suf¬ 
ficient  to  prove  a  constant  error.  [And  ws  should  intaijeot  hare  that  under  suoh 
circumstances  apparent  over-all  accordance  is  not  sufficient  to  prove  the 
abience  of  dally  oonetant  arrore  either.]  Tha  existence  of  an  accordance  an¬ 
alogous  to  a  'round  of  luok'  in  ordinary  Changes  Is  sufficiently  probable. .. . 
More  extensive  experience,  however,  may  give  greater  oonfidanoe  to  the 
assumption  ot  constant  errors. .  .first,  It  ought,  in  general  to  bo  established 
tnai  there  la  possibility  of  error,  oonetant  cm  ona  day  but  varying  from  day 
today...."  [Airy  1861,  p.  93.] 
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The  moat  uaaful  atatlatloal  tooli  (or  this  purpoaa  ara  tha  oontrol- 
ohart  techniques  of  tha  Induatrlal  quality  control  engineer.  If  In  such  a  alt- 
uatlon,  a  aarlaa  of  maaauramanta  obtained  by  measurement  of  e  tingle  quan¬ 
tity  a  number  of  times  on  aaoh  of  several  different  days  or  “occasions"  by 
a  particular  measurement  precast  it  plotted  In  tha  form  of.  a  control  ohart 
for  Individuals  [ASTM  1951,  pp,  76- 7B ,  and  pp.  101,  105 J ,  tha  Individual 
measurements  to  plotted  will  be  soon  to  consist  of  “saotrons"  Identifiable 
with  the  subsequences  (1)  corresponding  to  tha  respective  "oooaslons,  “ 

(l  ■  1,  2,  a, with  tha  measurements  within  taotlons  palr-wlte  closer 
together  on  the  average  than  two  measurements  one  of  which  comas  from 
one  section  and  the  other  from  another.  Such  a  series  of  measurements  Is 
clearly  “out  of  oontrol."  If  now  parallel  x-  and  R- charts  are  constructed 
from  these  data,  based  on  a  series  ol  samples  of  equal  alae  from  within 
the  respective  "nnoaslons"  or  “ sections “  only  .  l.e.,  excluding  means  R 
and  ranges  R  of  any  samples  that  “straddle"  two  oooeelons,  and  the  points 
on  tha  resulting  R-ehert  are  clearly  “out  of  control,  “  then  we  may  infer  the 
existence  of  day-by-day  components  of  error,  constant,  perhaps,  on  one  day, 
but  varying  from  day  to  day. 

If  points  on  the  R-ohart  constructed  as  described  are  “out  of  oontrol" 
also,  then  tha  measurement  operation  concerned  Le  In  a  completely  unstable 
condition  and  cannot  be  described  validly  as  a  “measurement  prooaas"  at  ell. 
On  tha  other  hand,  If  the  X-chart  is  "out  of  control, "  but  the  R-ohari  Is  "in 
control, "  then  wa  may  regard  the  measurement  pr^nass  as  being  In  a  state 
of  wlthln-occeslons  control.  (“It  le  usually  not  safa  to  concludo  that  l 
state  of  oontrol  exists  unless  the  plotted  points  for  et  least  2S  successive 
.'"bgroups  fall  within  the  3- sigma  oontrol  limits,  In  addition,  If  not  moru 
tnan  1  out  ot  35  successive  points,  or  not  more  then  2  nut  of  100,  felt  out¬ 
side  the  3-eiqma  oontrol  limits,  a  stats  of  buniicl  may  ordinarily  ba  assumed 
to  exist,"  f  A3A  1958o,  p.  lb,  ] )  In  such  a  situation  wo  pnatulata  ths  nxist- 
enoe  of  (at least,  conceptually)  different  limiting  means  (  for  tna  ,e*nsotlva 
"occasion*"  (1  ■  l,  2, . . and  a  common  withln-oooa atone  varianug  Cf*;  . 

An  unbiased  estimate  of  tha  wlth-ooca alona  standard  deviation  CTV. 
can  be  obtained,  If  dealred,  from  tha  average  i«»ga  R  used  In  construct.1  og 
the  R-chart,  by  meant  of  tha  formula 


(17)  unbiased  estimate  of  9*w  «  T(/dz 
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where  dj  ii  the  factor  given  In  the  d2  column  of  table  B2  of  [aSTM  1951,  p.  1  li] 
corresponding  to  the  eample  or  subgroup  sics  n  used  In  oona'truotlng  the 
R- chart. 

Alternatively,  If  desired,  an  unbiased  estimate  of  Cfz  can  be  ob- 
talned  directly  from  the  measurements  Involved  by  means  of  wthe  formula 


(IB) 


unbiased  estimate  of  O'  ^ 
w 


w 


l.  <xhj  -*h)2 

.h  e  Jl4jb  t  . 
k(n  -  1) 


where  Xj,j  denotes  the  |th  measurement  and  Xu  the  arithmetic  mean  of  the  u 
measurements  of  the  htu  subgroup,  respectively,  and  k  Is  the  number  of  sub¬ 
groups  Involved.  In  constructing  the  R-ohart, 

o.  Oomplex  or  Multistage  Control 

When  a  measurement  prooeee  le  not  In  e  state  of  simple  etatletloel 
control  that  satisfies  the  oriterts  of  wlthtn-oooesione  control,  thet  Is,  whan 
the  'k‘  chart  (and  control  ohert  for  Individuals)  are  olssrly  "out  of  control," 
hut  tm  25  or  mors  subgroup  rangss  plotted  on  the  R-nhart  exhibit  control , 
then  It  Is  usually  of  Importance  to  seoertaln  whether  the  meaeuroment  prooess 
oonoernad  Is  possibly  In  a  state  of  samnlflH  BE  imdUlttat  flQnkai. 

For  th<  h  purpose  four  or  more  measurements  from  each  of  et  lasst  20  different, 
oooemons  will  bs  needed,  Taklrci  one  sample  of  n  successive  measurements, 
(4*n  £>10),  frw.Vi  the  avullAbie  mostursmsntk  corresponding  lo  esoh  of,  say, 
MS 35)  different  "occasions,"  evaluate  the  arithmetic  mcanr  ft.  of  these 

samples,  (1  - 1,  2 . k),  end amna amuaaau *1 llffJiflBlIMi, 

mapsurtimants  construct  e  control  chart  for  these  "Individuals"  ani  parallel 
■R-  and  R-charte  as  dssertbed  In  [ASTM  1951,  Example  22,  p.  in] .  If  the 
points  plotted  on  those  three  control  cherts  exhibit  control,  thon  wt  "act 
lor  the  present  as  If"  the  measurement  process  oonoemed  Is  In  a  state  of 
Pom^eK  op  multistage  stetlstlcal  control  and  regard  the  limiting  means 
jj.  for  the  respective  "occasions,"  (1*1,  2, . . .)  as  bslng  In  a  state  ol 
simple  statistical  control  with  a  limiting  mean  / 1  and  variance  ffg  , 
termed  the  betwesn-ocoasloni  component  of  variance . 
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II  In  auoh  a  iltuation  we  war*  to  form  cumulative  antnmetlc  manna 
auoh  ae  (3)  of  the  aquarea  of  all  dlatlnot  dlffaranoaa  between  arbitrary  palra 
of  maaauramanta  from  within  aauh  of  tha  raapaotlva  "oooaitona,"  than  auoh 
cumulative  arithmetic  meane  of  aquaraa  of  dtff  arenoaa  would  almoat  auraly 
tend  to  2  In  tha  limit  aa  tha  number  of  palra  Inoludad  tanda  to  infinity, 

where  a  i,  lathe  "wlthtn*oooa  atone  variance "  mentioned  above  In  oonnaotlon 
with  "wltnln-occaalona  oontrol."  If,  on  the  other  hand  we  were  to  form 
almilar  cumulative  arithmetic  meant  of  the  aquarea  of  dlffaranoea  between 
arbitrary  palra  conitating  If  each  tnatanoe  of  one  meaaurament  from  each 
of  two  different  auction#,  then  auoh  a  cumulative  arithmetic  mean  of  equated 
difference  a  would  tend  almoat  certainly  to  21  g  *  +  (f£  )  aa  tha  number 

of  "tinea slona"  eampled  tanda  to  Infinity,  whara  o'?  la  the  above  mention- 
ad  "batwean-oooaatona  varlanoa,"  t.a.,  tha  variance  of  tha  limiting  meana 
/u.  i  for  tha  raapaotlva  "oooaatona"  about  their  limiting  main  pt  . 

If  in  utillaing  maaauramanta  from  a  measurement  prooeaa  that  la  In 
auoh  a  atata  of  ao.nplex  atatlatioal  control,  one  forma  an  average  that 

la  tha  arithmatlo  mean  of  a  total  of  N  ■  kn  maaauramanta,  oompoaad  of  n 
maaauramanta  from  each  of  k  different  "oooaatona, "  then  the  variance  of 
will  be 

2 

u>)  •^NZJo7m  ijcr'J  +  4*-) • 

1’iom  (19)  It  la  clear  that,  11  la  at  all  aliebla  compared  to  qr^  ,  than, 
for  fixed  N  ■  kn,  xN  will  V,ave  greater  preolalon  ae  a  determination  of  fj. 
whan  baaad  or  a  large  number  k  of  different  oooaaiona,  with  only  a  email 
number  n  of  meaaurvmenta  from  each  oooaalon.  Finally,  aettlng  k  -  1,  we 
aoa  that  tha  mean  of  n  meaaurementa  all  taken  on  tha  aame  ooo&tlor. 
oc.nildar.ad  aj-i-dityrmlnatlon  of.  tha  overall  limiting  maanu.  ha  a  an  avmall 
VmlftMB  or  g  -  0%  +  (  ay  nil  but  coneldtradai  a  determination  <A  ^u, 
the  limiting  mean  for  thaith  oooaalon,  It#  varlanoe  la  only  fjyn.  Ir>  other 
wurdv,  tha  "atandard  error"  of  a  mean  auoh  aa  Rt  la  not  unique,  hut  dependa 
on  the  purpoaa  for  which  it  la  to  ba  uaad. 
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An  unblaiad  estimate  of  tha  ovarall  standard  deviation  Vg:  of  tha 
arithmetic  mean  of  n  maaauramanta  taken  on  a  a  ingle  "oooaalon"  may  b« 
obtained  by  tha  procedure  of  formula  (1?)  above,  If  daalrad,  uaing  tha  average 
range  7  ejnployed  In  oonatructlng  the  R*ohart  oorreapondlng  to  the  groupa  of 
avaragea  xm  • 

H 

Alternatively,  an  unblaaed  aotlmate  of  the  overall  varlanae  ®i  can  be 
obtained  directly  from  the  mean*  xj  uaed  In  conatruoting  tha  R  -chart,  by 
uaing  the  formula 


k 


where  JT.  la  the  arithmetic  mean  of  the  n  lugneaalve  obaervatlona  from 
the  1th  ''occaalon, "  (1-1,2,  .  . ,  k)  and  IT  la  the  arithmetic  moan  of 

thoae  k  maana. 

The  foregoing  concept  of  a  atate  of  complex  nr  multlataca  atatlatloal 
control  can  be  extended  readily  to  more  oomplex  truly  "multlatage"  aituetlona 
unliving  three  or  more  "lavala"  or  random  variation. 

Finally ,  it  la  evident  from  the  foregoing  that  when  a  meaaurement 
process  la  In  a  it.ate  of  complex  or  multlatage  atatlatloal  control,  ihan  tha 
difference  between  two  Individual  maaauramanta  (or  tha  arithmetic  maana  of 
n  mbujuremanta)  oorramondlng  fn  two  different  "ocoaeluna"  will  Include  tha 
difference /u.  [  -  M>t,  uatwaan  the  limiting  mwers  coireapondlng  to  the  two 
particular  occasion*  Involved.  I.,  to  far  as  »uoh  a  comparlaon  la  regarded  ue 
a  unique  Individual  caia,  tha  difference  yu-t  -  M|,  la  *  fixed  constant  •  ml 
hence  a  ayatematic  error  affecting  this  comparlaon.  On  tha  other  hentl,  If 
the  nKferenca  between  these  two  Individual  meaeurementc  (or  those  two  ailth- 
motlc  means  )  la  regarded  only  as  a  typical  Inatanoa  of  the  outcomes  that 
might  ba  yioldad  by  tha  aame  meaaurement,  process  on  other  palra  of  oooaainn*, 
then  the  difference^  i  -  may  be  regarded  aa  a  random  oomponar,* 
having  a  waro  mean  and  variance  2  ffg  , 

If  goea  without  aaylng,  of  oouraa,  that  if  a  control-chart  analyala 
of  the  type  deacrlbed  above  la  undertaken  for  the  purpoae  of  ascertaining 
whether  tha  procaae  la  In  a  ntata  of  oomplex  control,  but  the  points  plotted 
on  the  R-chart  art  olaarly  "out  of  control,"  then  the  meaaurement  proceae 
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ooncer,i*a  cannot  be  regarded  ns  statistically  stable  from  occasion  to  ocoa- 
ston,  and  should  be  used  only  for  oomaaratlve  measurement  wlthtn-oooaalona. 
Even  when  suoh  a  measuramont  process  Is  used  solely  tor  comparative  meas¬ 
urement  within  "occasions, "  it  needs  to  be  ehown  that  comparative  meas¬ 
urements  of  fixed  dlfferenoes  are  In  a  etate  of  (simple  or  complex)  statistical 
control,  If  this  measurement  process  is  to  be  generally  valid  in  any  absolute 
tense.  Thus  In  the  oaie  of  the  thermometer  calibration  procedure  mentioned 
in  section  2,4  above,  one  needs  to  examine  the  results  of  repeated  meas¬ 
urement,  occasion  after  occasion,  of  the  difference  between  two  standard 
thermometers  and  flg  of  proven  stability  in  order  to  determine  whether 
the  process  la  or  le  not  In  a  state  of  simple  or  complex  statistical  oontrol. 

3.5.  Difficulty  ol  OhsraotarlslnB  thfl.Aoquraov  of  a  Measuremant 
Process.  Unfortunately,  there  dose  not  exist  any  single  comprehensive 
measure  of  the  accuracy  (or  i&pcauraoy)  of  a  measurement  process  (analogous 
to  the  standard  deviation  ae  a  measure  of  Its  Imprecision)  that  le  really 
satisfactory.  This  difficulty  stems  from  the  faot  that  "aoouraoy,"  Ilka 
“true  value, "  seems  to  Le  e  reasonably  definite  concept  on  first  thought, 
but  aa  soon  au  one  attempts  to  spsotfy  exactly  what  one  means  by  "aoouraoy" 
In  a  particular  situation,  the  concept  becomes  illusive!  and  In  attempting 
to  raaolvu  the  matter  one  oomae  face  to  feoa,  sooner  or  later,  with  the 
question!  "Accurate "  for  what  purpose  ? 

Gauss,  In  his  saoond  devslopmant  (1021-1623)  of  tho  Method  of 
Least  Squares  clearly  recognised  the  difficulty  of  characterising  sharply 
this  "accuracy"  of  any  particular  procedure: 

"Qutpps  queoetln  h*"?  per  rel  naturam  nltqutd  vaal  impllusi,  quod 
H untilin'1  r'.rs.'mi.  ;*•>'  nisi  pm  prlnctplum  aHquotenus  arbltrarlum  neqult. , . 
nequa  demonstratlonibus  mcPhamaUul*  dooldenda,  eed  llbero  tantum  orhltrln 
nimlttenda, "12  [anus&l023,  Pert  1,  Art,  6.1  1 


'll.  umgrateful  to  may  colleague  Fran*  Alt  for  the  following  literal  translation 
of  these  phrases! 

"For  this  question  Implies,  by  the  very  naturo  of  the  matUu  ,  jomethlng 
vague  which  c.innot  be  cioarly  delimited  except  by  somewhat  arbitrary  principle 
...  nor  oan  It  be  decided  by  mathematical  demoetratlons,  but  must  be  left 
to  mere  arbitrary  judgment." 
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Gauss  hlrnsslf  proposed  loo.  ott.  that  tho  mean  sauara  error 
of  a  procedure  —  that  Le,  o'  +  (/uu-  r  )2,  where  O'  le  ite  atandard 
deviation  i  and  fu.  -  t  ,  it«  bjyi  be  ueed  to  ohareoterlae  its  accuracy. 

Whtla  mean  acmare  nrrorts  a  uaeful  criterion  for  oomparlng  the  relative 
aoouraoiea  of  measurement  prooeiaei  differing  widely  In  both  pieolsion 
and  bias,  it  dearly  doea  not  "tell  the  whole  etory."  For  example,  if  one 
were  to  aoopt  the  principle  that  measurement  processes  having  the  same 
mean  square  error  were  equally  "aoourate, "  then  one  would  bu  obliged  to 
consider  the  measurement  prooeaaes  corresponding  to  the  three  ourvea 
shown  In  figure  3  si  being  of  equal  accuracy,  whereas  for  many  purposes 
one  would  regard  prooass  C  (portrayad  to  tho  right)  as  tha  "most  aoourate," 

In  spite  of  the  faot  that  tha  ohaneas  ot  scoring  a  "bull's  ayo"  or  "near  mlas" 
are  greater  In  tho  cane  of  procees  A  shown  In  the  upper  left. 

Alternatively,  If  one  were  to  say  that  two  measurement  processes 
were  equally  aoourate  when  exactly  the  same  proportion  P  of  tho  measure¬ 
ments  of  each  lay  within  ±  &  units  from  the  true  value,  then  for  P»0,5 
one  would  be  obliged  to  eey  that  tha  measurement  piooeeeea  oorrospondlng 
to  curves  e  and  d  In  the  lower  half  of  figure  1  ware  equally  auourote,  and 
that  the  measurement  prooass  corresponding  to  curve  e  In  the  upper  half  of 
tha  same  figure  was  slightly  more  aoourate  than  atther  e  or  d.  Or,  taking 
P  ■  0.68,  one  would  be  obliged  to  say  that  the  measurement  piooesees  cor¬ 
responding  to  the  three  ourvas  shown  In  ftgure  4  were  equally  aoourate, 

Trom  these,  and other  oases  easily  oonstruoted,  It  Is  readily  seen  that  It 
Is  unsatisfactory  to  regard  two  maasuremnnt  prnoateew  as  balng  equally 
aoourate  It  tho  samn  spsolfted  fraotlon  P  of  the  measurements  produced  by 
eanli  U*  within  tha  same  distance  from  tire  true  value. 

•  Thus  ons  is  led  oy  the  fore;.,  of  necessity  to  the  Inesoapabte  conclusion 
Ahat  ordinarily  (at  least)  two  numliui*  ar*  needed  to  adequately  characterise 
the  accuracy  of  a  measurement  prooass.  And  this  has  been  reaognleeu  by 
tho  American  Society  for  Testing  end  Motorists  In  thalr  raaant  recommend*- 
lions  I  A  STM  1901,  pp.  1789-1760 

"Generally  the  Index  of  accuracy  will  consist  of  two  or  more  different 
numburs,  since  the  concept  of  aoourauy  embraces  not  only  ths  oonoept  .•< 
precision  but  also  ths  idea  of  more  or  laea  consistent  deviation  from  the 
reference  level  (eystematlo  error  or  b'as),  It  Is  preferable  to  doacrlbe  noouraoy 
by  separate  values  Indicating  precision  and  biao," 
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The  fact  of  the  matter  Is  that  two  numbers  ordinarily  suffice  only  because 
the  "end  results"  of  measurement  and  calibration  programs  are  usually  averages 
or  adjusted  values  based  on  a  number  cf  Independent  “primary  measurements," 
and  such  averages  and  adjusted  values  tend  to  be  normally  distributed  to  a 
very  good  approximation  when  four  or  more  "primary  measurements"  are  In¬ 
volved.  This  is  Illustrated  by  figure  5,  which  shows  the  distributions  of 
individual  measurements  of  two  unbiased  measurement  processes  with  Identical 
standard  deviations  but  having  uniform. and  normal  "laws  of  error,"  te- 
upectlvely,  together  with  the  corresponding  distributions  of  arithmetic 
means  of  4  independent  measurements  from  these  respective  processes— 
these  latter  two  distributions  are  depicted  by  a  single  curve  because  the 
differences  between  the  tv/o  distributions  concerned  are  far  less  than  can 
be  resolved  on  a  chart  drawn  to  this  scale.  Since  both  of  the  processes 
concerned  are  unbiased,  "accuracy"  thus  becomes  only  a  matter  of  "precision"-- 
or  does  it?--both  curves  for  n  ■  1  have  the  same  standard  deviation,  do 
they  reflect  equal  "accuracy"?  Would  not  the  answer  depend  on  the  advan¬ 
tages  to  be  gained  from  the  small  errors  balanced  against  the  seriousness 
of  large  errors,  in  relation  to  the  purpose  for  which  a  single  measurement 
from  one  or  the  other  is  needed?  But  "th.,.  problem"  disappears  nicely  If 
averages  of  4  measurements  are  to  be  u.-r3d, 

4.  EVALUATION  OF  THE  PRECISION.  AND  OF  CREDIBLE  BOUNDS  TO 
SYTEMATIC  ERROR  OF  A  MEAS-UBEMEN.T  PROCESS, 

.'a  we  have  Just  seen,  two  numbers  are  ordinarily  needed  to  chaicictortao 
the  accuracy  a  measurement  process,  the  one  tndlca'.'ng  its  preolslon. 
and  the  other  Its  bias.  Ir.  ^settee.  how»v»r,  'die  bias  of  a  measurement 
process  Is  unknown  and  unknowable  because  the  "true  values"  of  quantities 
measured  arn  almost  always  unknown  and  unknowable.  The  principle  exception 
ts  when  one  is  measuring  a  difference  that  is  by  hypothesis  Identically  aero. 

U  rhe  bias  of  a  measurement  process  could  be,  and  were  known  exactly,  then 
ona  would  of  course  subtract  It  off  as  a  "correction"  and  thus  dispose  of 
entirely.  Sines  ordinarily  we  cannot,  expect  to  know  the  exact  magnitude 
o'  the  bias  of  a  measurement  process,  we  are  forced  In  practice  t.o  settle 
for  credible  bounds  to  Its  likely  magnitude— much  as  did  Steymng  and  the 
thief  in  chapter  VI  of  Kipling's  story,  Captains  Courageous:  "Steynlnq 
t uk,  him  for  the  reason  that  the  thief  tuk  the  hot  stove— bekase  for  iheru 
was  nothing  else  that  season" .  Consequently,  neither  the  bias  nor  the 
accuracy  of  any  measurement  process,  or  method  of  measurement,  can  ever 
be  known  tn  a  logical  sense.  The  precision  of  a  measurement  process,  how¬ 
ever,  can  be  measured  and  known.  (Compare  Demtng  |  1950,  p.  17  ]  .) 
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4.1.  Svftisiaugn  aLUlJLBigiiiga  g[  a  M«iut«meniPtftM8»-  in  the 
foregoing  we  have  stressed  that  a  measurement  operation  to  qualify  as  a 
measurement  process  must  have  attained  a  state  of  statistical  control;  and 
that  until  a  measurement  operation  has  been  "debugged"  to  the  extent  that 
it  haa  attained  a  state  of  statistical  control,  it  cannot  be  regarded  lu  any 
logical  sense  as  measuring  anything  at  all.  It  is  also  clear,  from  our  dis¬ 
cussion  of  the  control-chart  techniques  for  dstermlning  whether  m  any  given 
instance  one  is  entitled  to  "act  for  the  present  as  If"  a  state  of  statistical 
control  haa  been  attained,  that  a  fairly  large  amount  of  experience  with  a 
particular  measurement  process  is  needed  before  one  can  resolve  tne  ques¬ 
tion  In  the  affirmative.  Once  a  measurement  prooess  has  attained  a  state 
of  statistical  control  and  so  long  as  It  remains  In  this  state,  then  an  esti¬ 
mate  of  the  standard  deviation  of  the  process  oan  ba  obtained  from  the  data 
employed  in  establishing  control,  as  wa  have  indicatad  abova. 

Since  the  precision  of  a  measurement  process  refers  to,  and  la  deter¬ 
mined  by  the  oharaoteriitto  "closeness  together"  of  euooetslve  independent 
measurement!  of  a  single  magnitude  generated  by  repeated  application  of 
the  process  under  specified  conditions,  it  is  clearly  necessary  in  determining 
whether  a  measurement  operation  is  or  is  not  In  a  state  of  etatletloal  control, 
and  in  evaluating  Its  precision  to  be  reasonably  definite  on  what  variations 
of  procedure,  apparatus,  environmental  conditions,  observers,  opsretors, 
ate.,  are  allowable  in  "repeated  applications"  of  what  will  be  considered 
to  i>e  the  same  measurement  process  applied  to  the  measurement  of  the 
uarr.e  quantity  under  the  seme  conditions.  If  whatever  measure  of  the  pre¬ 
cision  and  bounds  to  the  bias  of  the  measurement  prooees  we  may  adopt 
ere  *n  provide  a  realistic  indication  of  the  aoouraoy  of  thla  process  in  prac¬ 
tice,  .hen  the  "allowable  variations"  must  be  of  sufficient  scope  to  bracket 
the  range  of  clrcum  its  none  commonly  met  in  practice.  Scientists  and  en¬ 
gineers  commonly  append  "probable  errors"  or  "standard  errors"  to  the  re¬ 
sults  of  their  experiments  and  tests,  these  measures  of  imprecision  are 
supposed  to  Indioate  the  extent  of  the  reproducibility  of  these  experiments 
or  testa  under  "essentially  the  same  conditions,"  but  there  are  great,  doubts 
whether  the  "probable  errors"  and  "standard  errors"  generally  preecntod 
actually  have  this  meaning.  The  fault  In  most  oases  le  not  with  the  statistical 
formulas  end  procedures  used  to  compute  suoh  probable  errors  or  standard 
error-  from  the  measurements  In  hand,  but  rather  with  the  limited  scope  oi 
the  "conditions"  sampled  in  takLng  the  measurements. 
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Concept  of  a  "Repetition"  of  a  Measurement 

As  a  vary  minimum)  a  " repetition' "  of  a  measurement  by  the  same 
measurement  process  should  "leave  the  door  open"  to,  end  In  no  way  Inhibit 
changes  of  the  sort  that  would  oeour  If,  on  termination  of  a  given  series  of 
measurements,  the  data  sheets  were  stolen  and  the  experimenter  were  to 
repeat  the  series  as  olosely  at  possible  with  the  same  apparatus  and  auxi¬ 
liary  equipment  following  the  tame  instructions.  In  contrast,  a  "repetition" 
by  the  same  method  of  measurement,  should  permit  and  in  no  way  Inhibit  the 
natural  occurrence  of  such  changes  as  will  ooour  if  the  experimenter  were 
to  mall  to  a  friend  complete  details  of  the  apparatus,  auxiliary  equipment, 
and  experimental  procedure  employed- * l. e. ,  the  written  text  specification 
that  defines  the  "method  of  measurement"  conoemed— and  the  friend,  using 
apparatus  and  auxiliary  equipment  of  the  same  kind,  and  following  the  pro¬ 
cedural  Instuctlons  received  to  the  bast  of  his  ability,  were  then,  after  a 
little  praotloa,  to  attempt  a  repetition  of  the  measurement  of  the  same  quan¬ 
tity.  Such  are  ths  extremes,  but  there  Is  a  "gray  region"  between  In  which 
there  is  not  to  be  found  a  sharp  line  of  demarcation  botween  the  "areas" 
corresponding  to  "repetition"  by  the  same  measurement  process,  and  to 
"repetition"  by  the  «um  aattBfljal  mimauntai;. 

Lot  us  consider  "repetitions"  by  ths  same  measurement  process 
more  fully,  Such  (repetitions  will  undoubtedly  be  carried  out  th  the  same 
place,  i.e.,  in  the  same  laboratory,  because  If  It  is  to  be  the  same  meas¬ 
urement  process,  the  very  same  apparatus  must  ho  used.  But  a  "repetition" 
cannot  bo  carried  out  at  the  same  time.  How  great  n  lapse  of  time  should 
bo  allowed,  nay  required,  between  "repetitions"  ?  Inis  is  a  crucial  ques¬ 
tion.  Student  gives  an  answer  in  a  passage  from  which  we  quoted  above 
[student  1917,  p.  ilSj: 

"Perhaps  I  may  be  permitted  to  restate  my  opinion  as  to  *be  bust 
way  of  judging  ths  Accuracy  of  physical  or  chemical  determinations. 

"After  considerable  experience  I  have  not  encountered  e,ny  determina¬ 
tion  which  is  not  influenced  by  the  date  on  whloh  it  Is  made;  from  this  it 
follows  that  s  number  oi  determinations  of  ths  same  thing  made  on  thq  same 
day  are  likely  to  lie  more  olosely  together  than  If  the  repetitions  had  been 
made  on  different  deye. 

"It  also  follows  that  if  the  probable  error  is  oaloulated  from  a  num¬ 
ber  of  obeervatlons  made  close  together  in  point  of  time,  much  of  the  secular 
error  will  be  left  out  and  for  general  use  the  probable  error  will  be  too  small, 
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"Where  then  tha  matartali  era  sufficiently  itabla  It  t*  wall  to  run 
a  numbar  of  datarmlnatlom  on  tha  aama  material  through  any  series  ol  routine 
determination*  whlon  have  to  ba  mad*,  tpraadlng  than  ovar  tha  whole  period." 

Another  Important  question  L*  t  Are  'repetition*"  by  the  aama  mass* 
urement  process,  to  ba  limited  to  repetition*  by  the  tarn*  observer*  and 
operators,  using  tha  same  auxiliary  equipment  (bottle*  of  reagents,  ato.); 
or  enlarged  to  Include  repetition*  with  nominally  equivalent  auxiliary  equip¬ 
ment,  by  various  but  equivalently  trained  obaarvera  and  operator*?  1  be¬ 
lieve  that  everyone  will  agree  that  substitution,  and  certainly  rep  aoement, 
of  bottle*  of  reagents,  of  batteries  as  source*  of  electrical  energy,  etc, , 
by  "nominally  aqulvalent  materials"  must  be  allowed.  And  any  calibration 
laboratory  having  a  large  amount  of  "bualnei*"  will  certainly,  In  the  long 
run  at  any  rate,  have  to  feoa  up  to  allowing  ohangee,  avon  raplaoemant  ol 
observers  and  operator*— and,  ultimately,  even  of  apparatus. 

Avery  crucial  question,  not  always  faced  squarely,  lsi  In  complete 
"repetition*"  by  tha  same  measurement  proocss,  ere  suoh  "repetitions"  to 
bo  limited  to  those  Interval*  ol  time  ovar  which  tha  apparatus  Is  used  "as 
is"  and  "undisturbed,"  or  extended  to  include  the  additional  variations  thet 
almost  always  manifest  themselves  when  the  apparatus  Is  disassembled, 
cleaned,  reassembled,  and  readjusted?  Unless  suoh  dl see sambly,  cleaning, 
reassembly,  and  readjustment  of  apparatus  Is  permitted  among  the  allowable 
variation*  effecting  a  "repetition"  by  the  earns  measurement  prooess,  then 
there  is  very  little  hope  of  achieving  setiefaotory  agreement  between  two 
or  more  measurement  processes  in  the  same  laboratory  that  differ  only  In 
their  Identification  with  different  places  of  apparatus  of  the  seme  bind.  In 
practice  It  Is  found  that  steMxtloal  control  uan  be  attained  ai.u  maintained 
under  such  a  owed  concept  of  "repetition"  only  through  the  use  of  referenoa 
standards  of  proven  stability-  furthermore,  by  thus  more  squarely  facing 
the  issue  of  the  scope  of  variations  allowable  with  respect  to  "repetitious" 
by  the  same  measurement  process,  we  shall  go  a  long  way  toward  narrowing 
fhe  gap  between  a  "repetition"  by  the  same  measurement  process  and  by 
the  same  method  of  measurement, 

.'.o  wc  have  said  before,  If  whatever  measures  of  the  precision  and 
bias  of  a  measurement  process  we  may  adopt  are  to  provide  a  realistic  In¬ 
dication  of  the  accuracy  of  this  process  In  praotloe,  then  the  "allowable 
variations"  must  be  of  sufftolent  scope  to  bracket  the  range  of  circumstances 
commonly  met  In  practice,  Furthermore,  any  experimental  program  that  almo 
to  determine  the  precision  and  systematic  srror,  and  thence  the  accuracy  of 
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a  measurement  process,  must  be  based  on  an  appropriate  random  sampling 
of  this  "range  of  olroumstanoas, "  If  the  usual  tools  of  statistical  analysis 
are  to  be  strictly  applicable.  Or  as  Student  put  It,  "the  experiments  must 
be  capable  of  being  consldsreu  to  be  a  random  sample  of  the  population  to 
which  the  conclusions  aro  to  be  applied.  Neglect  of  this  rule  has  led  to  the 
estimate  of  the  value  of  stati sties  which  Is  expressed  in  the  crescendo 
'lies,  damned  lies,  statistics' . "  f Student  1926,  o.  711. j 

When  adequate  random  sampling  of  the  appropriate  "range  of  clroum- 
stances"  Is  not  faaaible,  or  ovon  possible,  then  It  L>  necessary  to  compute, 
by  extrapolation  from  available  date,  a  more  or  lesa  subjective  estimate  of 
the  "preaislun"  of  the  end  results  of  a  moesurement  operation,  ,to  aerve  as 
a  substitute  for  a  direct  experimental  measure  of  their  ''reproduolblllty. " 
Youden  [l962d]  calls  this  "approach  the  'paper  way'  of  obtaining  an  eatlmate 
of  the  [precision] . "  Its  validity,  if  any,  "la  based  on  subject-matter  know¬ 
ledge  and  skill,  Jgenerel  Information,  and  intuition— but  not  on  statistical 
methodology"  [Cochran  et  el.  1953,  p.  693  ], 

b,  Some  Examples  of  Realistic  "Repetitions" 

As  Student  remarked  [l917,  p.  4is],  "The  beet  way.of  Judging  the 
accuracy  of  phyaioal  or  chemical  determination  . , .  [when]  the  materials 
are  sufficiently  stable. . .  Is  ...  to  run  e  number  of  determinations  on  the 
same  material  thru  eny  series  of  routine  determinations  which  hava  to  bo 
made,  spreading  them  over  the  whole  period."  To  this  end,  as  well  a~  to 
o'-ovlde  an  overall  check  on  procedure,  on  the  stability  of  reference  standnrds, 
-id  to  guard  against  mistakes,  It  is  common  practice  in  many  calibration 
procedures,  to  utilise  two  or  more  reference  standards  as  part  of  the  regular 
calibration  procedure . 

The  calibration  procedure  for  llauld-ln-qlesa  thermometer*,  referred 
to  in  section  2.4  above,  la  a  case  In  point.  A  measurement  of  the  difference 
between  the  two  standards  Sj  and  S2  is  obtained  as  by-product  of  ihu  cali¬ 
bration  of  the  four  test  thermometer*  ,  Tj  ,  T3,  and  T4  In  terms  ot  the 

(corrected)  readings  of  the  two  standards.  It  Is  such  remeasurements  c£ 
the  difference  between  a  pair  of  standard  thermometer*  from  "occasion"  to 
"occasion"  that  oonitltutes  realistic  "repetitions"  of  the  calibration  proce¬ 
dure.  The  date  yielded  by  these  "repetitions"  are  of  exactly  the  type  needed 
(a)  to  ascortotn  whether  or  not  the  prooess  Is  In  e  state  of  statistical  con¬ 
trol)  end  If  so,  (b)  to  determine  Its  overall  standard  deviation, 
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Similarly,  In  the  calibration  of  laboratory  standards  of  mass  at  the 
National  Bureau  of  Standards,  "known  standard  weights  are  calibrated  alde- 
by-slde  with  [the]  unknown  weights"  [Aimer  at  al. ,  1362,  p.  33 ]  .  Indeed, 
weights  whose  values  are  otherwise  determined  "are  not  said  to  nave  been 
'calibrated'.  That  term  Is  reserved  for  measurements  based  on  at  laast  two 
mass  standards."  [loo.  olt,,  p.  43,1  In  the  specimen  work  sheets  exhibited 
by  Aimer  at  al. ,  the  auxiliary  standards  Involved  are  those  from  the  Bureau's 
"NH  eerLes"  of  reference  standards  known  by  the  designations  NHSO,  NK20, 
NHlOj  respectively.  It  Is  tho  measurements  obtained  In  routine  calibrations 

of  the  differences  between  the  values  of  these  standards  and  thetr  accepted 
values  that  not  only  provide  valuable  checks  on  day-to-day  procedure,  but 
alto  serve  at  the  basis  for  determination  of  the  overall  standard  deviation 
of  this  calibration  process. 

A  third  example  Is  provided  by  the  method  followed  at  the  National 
Bureau  of  Standards  for  testing  alternating-current  watthour  metars .  whloh 
has  bean  described  In  some  detail  by  8plrtkt  and  Zapf  [1354]  ,  Four  refer¬ 
ence  watthour  matters  are  involved.  One  of  these,  termed  Athe  Standard 
Watthour  Meter, "  Is  located  In  the  device  portrayed  In  figure  1  of  the  paper 
by  Spinks  and  Zapf.  The  other  three  are  located  In  a  terperatura- controlled 
ohblnet .  A  "test"  of  a  watthour  meter  sent  to  the  Bureau  Involves  not  only 
a  t  umparlton  of  this  watthour  meter  with  the  Standard  Watthour  Meter,  but 
also  comparisons  of  eaoh  of  the  Comparison  Standard  Watthour  Meters  with 
the  Standard  Watthour  Meter.  It  Is  from  the  data  yielded  by  these  Interoompar- 
Isons  of  the  Standard  Watthour  Meter  and  the  Comparison  Standard  Watthou. 
Meters  that  the  standard  deviation  of  this  test  procedure  Is  evaluated.  Spinks 
and  Zopf's  section  of  "Precision  and  Accuracy  Attainable"  Is  notable  for  tte 
exceptional  lucidity  .nr.  well  as  lor  its  completeness  with  respect  to  relevant 
details. 


Some  additional  examples  of  realistic  "repetitions"  are  dlsouseeu 
by  You den  [196’oj , 
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1.2.  Treatment  of. Inaocuracv  Due  to  Systematic  Errors  of I  Assignable 
OrloLne  but  of  Unknown  Magnitudes.  A*  we  remarked  in  Motion  3.3b  abbvu. 
the  eyitematlo  error  of  e  meaeurement  process  will  ordinarily  have  both  con¬ 
stant  and  variable  components.  For  convenience  of  exposition,  It  Is  cus¬ 
tomary  to  regard  the  Individual  components  of  the  overall  systematic  error 
of  a  measurement  or  calibration  process  as  elemental  or  constituent  "sys¬ 
tematic  errors"  and  to  refer  to  them  simply  as  "systematic  errors,"  for  short. 
Included  among  suah  "systematic  errors"  affecting  a  particular  measurement 
or  calibration  process  are:  "...  all  those  eirors  which  cannot  be  regarded 
as  fortuitous,  as  partaking  of  the  nature  of  ohanoe.  They  are  characteristic 
of  the  system  Involved  in  the  work:  they  may  arise  from  errors  In  theory  or 
In  standards,  from  Imperfections  In  the  apparatus  or  in  the  observer,  from 
false  assumptions,  eto.  To  them,  the  statistical  theory  of  error  does  not 
apply,"  [Dorsey  1944,  p.  6;  Dorsey  and  Eisenhart  1983,  p.  104.1 

The  overall  systematic  error  of  a  measurement  process  ordinarily 
consists  of  elemental  "systematic  errors"  duo  to  both  assignable  and  un¬ 
assignable  causes.  Those  of  unknown  (not  thought  of,  not  yet  Identified, 
or  as  yet  undiscovered)  origin  ore  always  to  be  feared;  allowances  oan  La 
made  only  for  those  of  recognised  origin. 

Since  the  "known"  systematic  errors  affeotlng  s  measurement  process 
asorlbable  to  specific  origins  are  ordinarily  determinate  In  origin, only, 
their  Individual  values  ordinarily  being  unknown  both  with  respedt  to  sign 
and  magnitude,  It  is  not  possible  to  evaluate  their  algebraic  sum  and  there¬ 
by  arrive  at  a  value  for  the  overall  systematic  error  of  the  measurement 
process  oonoamed.  In  consequence,  It  is  naossssry  to  arrive  at  bounds 
lor  each  of  the  Individual  components  of  systematic  error  that  may  be  expected 
to  yield  rmnnng'tgLblu  contributions,  end  then  from  these  bounds  arrive  at 
credible  bounds  to  thair  combined  effect  on  the  measurement  process  oon- 
oarnad.  Both  of  these  steps  are  fraught  with  difficulties. 

Determination  of  reasonable  bounds  to  the  systematic  error  llknlv 
to  be  contributed  by  a  particular  origin  or  assignable  cautc  nsoossarlly 
involves  en  element  of  judgment*  and  the  limits  cannot  bs  sot  In  sxaotitudo. 

By  assigning  ridiculously  wtds  limits,  ons  could  bo  praotloally  cabala  that 
the  aotual  error  due  to  a  particular  oause  would  never  He  outside  o i  these 
limits.  But  such  limits  are  not  likely  to  ba  vary  helpful.  The  narrower  the 
range  batwaan  the  assigned  limits,  the  greater  the  uneasiness  one  feels 
that  the  si  signed  limits  will  not  Include  whatever  systematic  error  Is  con¬ 
tributed  by  the  cause  In  question.  But  a  daotsion  has  to  ba  mada;  and  on 
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the  basis  of  theory,  other  related  measurements,  a  careful  study  of  the 
situation  in  hand,  especially  its  sensitivity  to  small  changes  in  the  factor 
concerned,  and  so  forth,  "the  experimenter  presently  will  feel  justified  in 
saying  that  ha  feels,  or  believes,  or  Is  of  the  opinion, "  that  the  systematic 
error  due  to  the  particular  souree  in  question  does  not  exoeed  such  and  suoh 
limits,  "meaning  thereby,  since  he  makes  no  claim  to  emnlsoienoe,  that  he 
has  found  no  reason  for  believing"  that  it  exoeeds  these  limits.  In  other 
words,  "nothing  has  corns  to  light  in  the  course  of  the  work  to  indlgste" 
that  the  systematic  error  oonoamed  lies  outside  the  stated  range.  [Dorsey 
1944,  pp,  9-10;  Dorsey  and  Elsenhert,  L9S3,  pp.  105-107,1  L 

This  being  done  to  each  of  the  reoognlsad  potential  sources  of  sys¬ 
tematic  error,  the  problem  remains  how  to  determine  credible  bounds  to 
their  combined  effect.  Before  considering  this  problem  in  detail,  It  will  be 
helpful  to  digress  for  a  moment,  to  ooneider  an  Instructive  example  relating 
to  the  combined  affect  of  constant  errors  in  an  everyday  situation. 

a.  .In  Instructive  Example 

Consider  the  hypothetical  situation  of  an  individual  who  is  comparing 
his  checkbook  balance  with  his  bank  statement.  To  this  end  he  needs  to 
know  the  total  value  of  his  checks  outstanding.  Loathing  addition,  or  per- 
har.j,  simply  to  save  time,  he  adds  up  only  the  dollars,  neglecting  the 
cents,  and  thus  errlvee  at  a  total  of ,  say,  $312,  for  20  oh eoks  outstanding. 
Adding  a  correction  of  50  oents  per  oheok,  or  $10  in  all,  he  takes  $322  as 
his  estimate.  Within  what  limits  should  he  consider  the  error  of  this  estimate 
to  lie  f 


The  round-off  ttnor  oanno*  sxoeed  *50  oents  psi  oheok,  so  that 
barring  mistakes  in  addition,  he  con  be  absolutely  oertaln  that  the  total 
error  of  his  estimate  doe*  not  sxosed  *$10.  But  these  are  extremely  pes¬ 
simistic  limits:  they  correspond  to  svsry  oheok  being  in  error  by  the  mex- 
itnum  possible  amount  end  Ml  in  the  seme  direction.  (Actually  the  maximum 
possible  positive  error  le  49  oents  per  check  or4$9.80  In  ell.) 

To  be  aoneervstlve,  but  not  so  pessimistic,  one  might  "allow"  r, 
maximum  error  of  *50  oents  psr  oheok,  but  ooneider  it  reasonable  to  rogeid 
their  signs  at  bslng  aqually  likely  to  be  plus  or  minus.  In  this  way  on* 
would  be  lad  to  oonoluda  "with  probability  0,95"  that  the  total  error  lies 
between  ±$7.00;  or  "with  probability  0.99,  "  bstwsen  *$8.00,  as  shown  in 
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the  column  headed  "binomial  In  table  l,  for  n  ■  20.  The  “aaving"  by  this 
procedure  It  clearly  not  great. 

Alternatively,  one  might  consider  it  to  be  more  "realittlo"  to  regard 
the  Individual  errors  aa  Independently  end  uniformly  distributed  between 
-50  cent i  and  +50  cents,  concluding  "with  probability  0.95"  that  the  total 
error  doea  not  exceed  ±82,53;  or  "with  probability  0.99,"  It  not  greater 
then  ±$3.33--at  shown  In  the  column!  under  the  heading  "uniform"  In  table 
1 .  It  lo  clear  that  a  considerable  reduction  In  the  estimate  of  the  total  error 
Is  achieved  by  this  approach. 

Strictly  speaking,  the  foregoing  analyses  via  the  theory  of  probability 
are  both  Inapplicable  to  the  problem  at  hand:  each  round-off  error  Is  a 
fixed  number  between  ±50  cents,  and  their  sum  Is  a  fixad  numbar  between 
±$10.  If  It  ware  true  that  round-off  errors  In  such  oases  ware  uniformly 
distributed  between  ±50  conts,  than,  if  one  mode  a  habit  of  evaluating  limlta 
of  error  according  to  this  procedure,  one  aould  expect  the  limits  of  error  so 
calculated  to  Include  the  true  total  error  m  95  percent,  or  99  peroent  of  the 
Instances  In  whloh  this  procedure  was  used  In  the  long  run.  Round-off 
errors  In  such  oases  ere  almost  certainly  not  uniformly  distributed  between 
±50  cents,  (Many  Items  ere  priced  these  days  at  $2.98  etc. ,  and  this  will 
distort  the  distribution  of  the  oents-portlon  of  one's  bills  but  added  tales 
t.ixes  no  doubt  havt  a  "smoothing"  effect.) 

Nevertheless,  1  believe  that  you  will  agree  that  If,  In  the  hypothetical 
case  under  dltouaslon,  the  checkbook  balance,  with  an  allowance  of  $322 
to:  checks  outstanding,  fatlnd  to  agree  with  the  bank  statement  to  within 
$2,33  (or  $3.33),  our  "frlonr1."  would  do  well  to  check  I  etc,  die  matter  more 
thoroughly.  Ar,d,  alternatively,  if  his  checkbook  balanoe  so  adjusted,  and 
the  bank  statement,  agreed  to  with  $2.53  (or  $3.33),  It  would  be  reasonably 
"uate"  for  him  to  "act  for  the  present  as  If"  hls  balanoe  and  the  bank  siii'.r- 
meni  were  In  agreement,  (See  Eisenhart  Q947a,  p.  218J  for  discussion  of  a 
similar  example  relating  lo  computation  with  logarithms,) 

b.  Combination  of  Allowanoos  tor  systematic  Errors 

The  foregoing  example  suggests  that  a  similar  procedure  b«  used 
for  arriving  at  credible  limits  to  the  likely  overall  effeot  of  systematic 
errors  due  to  a  number  of  different  origins.  A  numbar  of  additional  difficulties 
confront  us,  howeve: ,  In  this  case.  To  begin  with,  In  view  of  the  Inexact¬ 
ness  with  which  bounds  can  ordinarily  be  placed  on  each  of  the  Individual 


■»tor»o6a5o*-iNf*M$c>'-<»»>'Oe4 

OOCiOO'-'r-.-J-i.-il-.i-iC'ie'iCipjm 


3$&;3f38$888{«$S£&S8 

ddddodddo'rtHrlHHciNN 


’*j*^HC^o>’tfQOoM«^'0505ii^5r>:»o<2 

q&OS'-«?l,9,55K>-00050‘^,o6C^»oO“v'5i 

oo’-4H'-5»-4^^r4csc4c^coc^v^Tr^ 


?gffi®28S8!?:882:3$2Sg 

oooG>fi*-5T-5i-?r4»^i-^c4c4csicotO€0 


-H 

a  a? 

B  ° 


3E3328$38$S88gS5SS 

O© e  m  •-«  "-i r4 r-«  -h  i~i c4 d  oi  ei  e*5  e*S ■* 


88gg$88SgSZ?S8SS32 

ddddooVir»Mt-««iHc4Ncic4w 


882582SS2S88KS2SS 

oo-^~i<-«'-'^c4rioic'icT>«-rVioio 


ssessasssssssssssss 

oc>o'-H*~<»-<«-5»-<»-<^Hc4c4cdc»3'y*'*»5 


I  — 

«  -H 

I  o 

I  2 

i  ® 


28883828888888388 

o^i^e4ciroriMi'*'»'Oo6o>’-'’«!^C} 


loSSSSaSS^SSSSS^SSS 

o^*-<c4c4c4coccc»5M:‘oi^cooco<d05 


SSSSSSSSSSSs^SSSS 


HMrtfifl«OMJOOO«Ojf)gCGO 

T-H*“»CjCSC*3*«»«3tO 


52B 


Design  of  Experiments 


components  ol  eyatamatto  eiror,  It  is  not  possible  to  say  with  absolute 
certainty  that  thalr  combined  etlect  lies  between  tho  sum  of  the  positive 
bounds  and  the  sum  of  the  negative  bounds. 

Soeond,  even  If  it  were  possible  to  scale  the  situation  so  that  the 
bounds  for  eaah  of  the  components  of  systematic  error  was  the  same.  say. 
i  A ,  there  would  still  remain  the  problem  of  translation  Into  an  appropriate 
probability  calculus.  Most  persons  would,  I  believe,  regard  the  "binomial" 
approach  (corresponding  to  equal  probability  of  maximum  error  In  either 
direction),  as  too  pessimistic)  and  the  approaoh  via  a  uniform  distribution 
of  error,  aa  a  bit  conservative,  on  the  grounds  that  one  Intuitively  feela 
that  the  Individual  errors  sre  somewhat  more  likely  to  lie  near  the  oentera 
than  near  the  anda  of  their  reapeotlva  ranges.  Therefors,  one  might  attempt 
to  simulate  thle  "feeling"  by  eeaumlng  the  "lew  of  error"  to  be  an  leoaoelea 
triangle  centered  at  sero  end  ends  at  ±A  (  or,  more  daringly,  by  eeaumlng 
the  "lew  of  error"  to  bo  approximately  normal  with  A  corresponding  to  2  "o' " 
or  even  3  "  O' 


Unfortunately  whatever  "probability  llmtta"  may  be  placed  upon  the 
combined  effeota  o'  aeveral  independent  eyetemetlo  errors  by  theee  proce¬ 
dural  ere  quite  ;  uitlve  to  the  assumption  made  at  this  stage,  aa  Is  evident 
from  table  1 .  Therefore,  anyone  who  uses  one  of  those  methods  for  the 
"combination  of  errors"  should  tndloata  explicitly  which  of  these  (or  an 
altc-nativa  method)  he  hee  used.  When  (a)  the  number  of  uyetomatto  errors 
to  be  combined  Is  larqs,  (b)  the  respective  ranges  ere  approximately  equal 
In  also,  and  (a)  one  feele  "fairly  sure"  that  the  individual  errors  do  not  fall 
outeion  of  their  respective  ranges,  then  my  personal  foaling  Is  that  tho 
"unifcim"  method  Is  probably  a  was  bit  oonservattve  but  "safe")  tho  tilengular 
method  Is  a  bit  "Umj  dsiing":  the  normal  method  with  "&'•  -  A  /3  ordinarily 
"much  too  daring "j  but  the  normal  method  with  "O'"  ■  A/2,  probably  "not 
too  verlng."  'When  (b)  and  (o)  hold  but£  la  email,  than  It  will  probably 
bo  sate  to  uea  tho  "uniform"  mathod  with  "A  "  takon  aqual  to  tha  average 
of  tha  individual  ranges,  other  oases,  o.g.,  when  n  Is  large  but,  say, 
one  ut  two  of  the  ranges  is  (ere)  much  larger  than  the  others  and  lnnd(i)  to 
dominate  the  situation,  requires  speoial  consideration  which  Is  beyond  the 
scop*  o'  the  present  paper, 

4.3.  Expression  of  the  Inaccuracy  of  a  Measurement  Process.  By 
whatever  means  credible  bounds  to  tho  likely  overall  systematic  error  of  the 
measurement  prooaas  are  obtained  thwy  should  not  be  combined  (by  simple 
addition,  by  "quadrature , "  ot  otherwise)  with  an  experimentally  determined 
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measure  of  Its  standard  deviation  to  obtain  an  overall  Index  of  Its  accuraoy 
(or,  more  correctly,  of  its  inaccuracy) .  Rather  (a)  the  standard  deviation 
of  the  prooosa  and  (b)  credible  bounds  to  its  systematic  error  should  be 
stated  separately,  beoausa,  as  we  showed  In  flours  3.  a  measurement  process 
having  standard  deviation  0.25  and  a  bias  A  «V  15/1 6  ■  0,97  Is  for 
most  purposes  "more  accurate"  than  a  measurement  process  having  aero 
bias  and  standard  deviation  1,  to  that  a  process  with  O'"  0.25  and  a 
bias  less  than  ±0.97  will  a  fortiori  be  "more  aoourate. " 

Finally,  it  the  uncertainties  In  the  assigned  value  of  »  notional 
standard  or  of  some  fundamental  constant  of  nature  (e.g. ,  In  the  volt  as 
maintained  at  the  National  Bureau  of  Standards,  or  In  the  speed  of  light  £, 
or  In  the  acceleration  of  gravity  g  on  the  Potsdam  basis)  Is  an  Important 
potential  source  of  systematlo  error  affecting  the  measurement  process,  no 
allowanoe  for  possible  systematlo  error  from  this  source  should  be  Included 
ordinarily  In  evaluating  overall  bounds  to  the  systematlo  error  of  the  meas¬ 
urement  process.  Since  the  error  concerned,  what  ever  It  Is,  affects  all 
results  obtained  by  the  method  of  measurement  Involved,  to  Include  an 
allowanoe  for  this  error  would  be  to  make  everybody's  results  sppear  unduly 
Inaoourete  relative  to  eaohi  other.  Instead,  In  suoh  Instances  one  ahould 
state  (a)  that  results  obtained  by  the  measurement  prooeis  concerned  ere 
In  terms  of  tha  volt  (or  the  watthour,  or  the  kilogram,  eta.)  Nes  maintained 
at  the  National  Buiuau  of  Standards"  [MoNlsh  end  Gameron  I960,  p,  102j, 
or  "correspond  to  ths  speed  of  light  o  ■  2.997923  X  tol 0  cm/aao.  exactly. 11 
■ay;  and  (b)  that  the  Indicated  bounds  to  the  systematic  error  of  the  process 
ary  exclusive  of  whatever  errors  may  be  present  from  this  (or  thsse)  soutoe(s). 
G'ven  such  Information,  sxperts  can  make  suoh  additional  all  wanrss,  ns 
may  bs  needed,  in  fundamental  scientific  work;  end  comparative  measurements 
within  solenoe  and  industry  wttnln  the  United  States  will  not  sppoar  to  bs 
less  aoourate  than  they  very  likely  are  for  the  purposes  for  which  they  are 
tc  be  used, 


It  is  a  pleasure  to  acknowledge  the  technical  assistance  of  Janaoa 
A.  Spaokman  In  several  phases  of  the  preparation  of  this  paper. 
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INTRODUCTION,  The  original  paper  presented  at  the  Ninth  Conference 
on  the  Design  of  Experiments  in  Army  Research,  Development  and  Testing 
consisted  of  the  presentation  of  a  program  for  the  statistical  study  of  data 
derived  from  troop  training  flights  of  Nike  Hercules  and  Hawk  missiles, 
followed  by  a  discussion  and  interpretation  of  the  results  of  this  study  with 
particular  emphasis  on  reliability,  failure  modes,  and  the  analysis  of  miss 
distance  data.  The  major  portion  of  the  paper  contained  classified  Informa¬ 
tion  which  cannot  be  presented  here.  For  access  to  the  classified  material, 
which  consists  primarily  of  the  results  of  the  statistical  study,  the  reader 
is  referred  to  Ballistic  Research  Laboratories  Memorandum  Report  No, 

1487,  July  1963,  Reliability  and  Performance  of  the  Nike  Hercules  Missile 
System)  and  Ballistic  Research  Laboratories  Memorandum  Report  No.  1  SI 3, 
October  1963,  Reliability  and  Performance  of  the  Hawk  Miesile  System) 
both  of  which  are  classified  confidential.  Some  classified  information  which 
was  presented  at  the  conference  on  the  analysis  of  miss  distance  data  for 
Nike  Hercules  has  not  been  published  but  will  appear  in  a  forthcoming  BRL 
Memorandum  Report,  The  material  which  follows  is  an  abridged  paper 
Including  the  unclassified  portions  of  the  original  paper. 

PURPOSE.  The  statistical  study  of  reliability  and  accuracy  ai  surface- 
to-air  micelles  Is  yiart  of  the  overall  surveillance  program  to  evaluate  the 
stockpile  of  Army  surface-!'' -air  missiles.  Ths  data  ussd  in  this  portion 
of  the  program  is  derived  from  the  Army's  Package  Training  r^neram  and 
Annual  Service  Practice  Program.  The  results  of  this  study  will  be  into 
grated  with  other  phases  of  the  surveillance  program  for  the  accomplish¬ 
ment  of  ths  following  objectives: 

1.  To  provide  a  continuing  evaluation  nf  ths  saftty,  reliability, 
and  performance  of  the  stockpile  of  missiles, 

2,  To  provide  advance  Information  concerning  any  degradation 

in  the  safety,  reliability,  or  performance  of  theae  missile*  so  that  timely 
action  can  be  taken  to  maintain  the  required  Inventory  levels  and  performance 
standard!  by  either  repair  or  replacement. 

Other  benefits  which  can  be  realised  from  the  analysis  of  troop  training 
firings  are  such  things  ss: 
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1.  The  accumulation  of  basic  knowledge  of  the  characteristics  of 
current  aurface-to-air  missiles, 

Z.  An  increase  in  our  general  fund  of  knowledge  about  missile 
systems  reliability,  and 

3,  A  basis  for  the  prediction  of  expected  reliability  of  future 
missile  systems. 

All  of  these  benefits  will  be  valuable  In  formulating  test  programs 
and  analysis  programs  for  future  missile  systems, 

rilUCRAM  FOR  STATISTICAL  ANALYSIS.  The  first  step  in  the 
analysis  of  the  data  consists  of  establishing  standards  for  classifying 
each  missile  firing  as  either  successful  or  unsuccessful.  With  such  a 
definition  in  hand,  an  overall  reliability  figure  for  the  entire  time  period 
under  consideration  can  be  obtained.  However,  in  order  to  study  reliabil¬ 
ity  as  a  function  of  time,  it  1s  helpful  to  construct  a  control  chart  for  the 
percent  of  successful  flights,  The  data  is  divided  into  groups,  ordered 
in  time,  of  n  missiles  each  and  the  percent  success  is  computed  for  each 
group,  The  number  of  missiles  to  Include  in  each  group  is  arbitrary,  but 
should  be  large  enough  so  that  random  runs  of  successes  or  failures  will 
not  adversely  affect  the  group  reliability,  Group  sisee  of  20  to  40  missiles 
wlm  :‘.<r  control  limits  have  been  satisfactory  for  this  application.  When 
this  data  is  plotted  on  a  control  chart  It  la  relatively  easy  to  observe 
whether  any  significant  changes  in  reliability  have  occurred  with  respect 
to  time.  Groups  out  of  control  or  trending  up  or  down,  will  indicate  a 
need  for  a  closer  examination  of  the  data  to  determine  the  cauee.  Ae 
addu.onal  data  becomes  available  it  may  be  necessary  to  recompute  the 
control  limits  or  to  compute  different  control  limits  for  different  groups 
depending  on  the  lectors  involved  for  each  particular  group.  Differences 
augmented  by  the  control  chart  can  bs  tested  for  Independence  hy  the  use 
ol  contingency  table*  and  the  distribution, 

In  the  analysis  of  failure  modes  for  a  missile  system,  it  is  generally 
satisfactory  to  classify  failures,  as  a  percent  of  total  missiles  launched, 
in  u  two-way  table  of  failure  categorise  and  time  intervals.  With  such  a 
classification  It  may  bs  possible  to  show  significant  changes  in  failure 
modes  corresponding  to  changes  in  reliability  suggested  by  the  control 
chart  for  percent  success.  This  table  will  also  show  the  particular  fail¬ 
ure  categories  or  sob-categories  which  contribute  the  moat  failures  to 
the  missile  system  in  any  given  time  Interval.  The  magnitude  of  the  time 
intervals  will  depend  upon  the  quantity  of  data  available  for  analyeia  but 
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the  intervals  do  not  necessarily  have  to  be  equal.  The  existence  of 
significant  differences  in  failure  rates  between  two  time  intervals  or 
between  several  time  intervals  can  be  tested  by  the  use  of  contingency 
tables  and  the  ”XZ  distribution. 

The  analysis  of  miss  distance  data  for  a  missile  system  generally 
consists  of  presenting  a  frequency  distribution  of  miss  distance  and  a 
control  chart  for  the  mean  miss  distance  and  for  the  standard  deviation 
of  miss  distance.  For  the  control  chart  the  miss  distances  are  ordered 
in  time  and  divided  into  groupa  of  n  missiles  each.  Oroup  elaee  of  10 
to  23  miaailea  with  3o-  control  limit*  have  been  satisfactory  for  this 
application.  When  group  means  and  standard  daviatlone  are  plotted  on 
the  control  chart,  it  will  be  readily  observabla  if  any  of  the  groups  are 
out  of  control.  Differences  between  group*  or  among  several  group* 
can  be  tested  for  (ignificance  through  the  use  of  a  t-test  or  one-way 
claeeification  analyst*  of  variance. 

In  addition  to  the  above  ana'y  its  of  int>  ■  dietsnee,  It  ia  daalrabl*  to 
examine  the  frequency  distribution  of  mi  sc  distance  to  determine  if  it 
can  be  characteriaed  by  eom*  theoretical  fraquoncy  distribution.  Several 
typical  distribution!  are  fitted  to  the  observed  data.  The  "goodnea*  of 
fit"  test  la  used  in  the  comparison  of  the  observed  distribution  with  the 
theoretical  distribution  lo  test  the  hypothesis  tbaL  a  particular  distribution 
ate  the  observed  data. 

Another  point  recently  considered  in  connection  with  the  analysis  of 
n.iss  distance  data  was  tha  possible  existence  of  a  relationship  between 
miss  distance  and  the  tavg.n  parameters  at  the  time  of  intercept.  This 
problem  ii  Investigated  through  a  linear  regression  analysis  of  miae 
distance  a*  a  (unction  of  the  rang.,,  altitude,  and  velocity  of  the  target 
at  intercept,  The  equation  used  In  thia  etudy  is  of  the  form 

y  -  0L+  /31X1  +  02X2  +  £3X3 

where  y  t*  mies  diatance,  Xj  is  range,  X^  is  altitude,  and  in 
velocity.  The  parameters  0.,  £>v  &v  and  £  are  true  values,  to 

be  estimated  by  a,  bj,  b.,,  and  b,}  respectively.  The  method  oi  least 

squares  is  employed  uhich  minimises  the  sum  of  square*  of  the  devia¬ 
tion*  of  the  (tom  the  hyperplane 
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v  ■  .  *  b,*,  ♦  b2x;  *  b3x2 

The  b  values  are  then  tested  for  ilgnlfleant  effect*  through  the  uae  of 
a  t-tc»t, 

RESULTS  OF  THE  STATISTICAL  ANALYSIS.  A ■  mentioned  earlier 
In  this  document,  the  reault*  01  the  statistical  analysis  are  clasotfied 
and  cannot  be  presented  here,  However,  it  should  be  pointed  out  at 
this  time  that  the  statistical  techniques  employed  in  this  program  made 
a  substantial  contribution  toward  tha  accomplishment  of  the  objectives  of 
the  overall  surveillance  program  to  avsluate  the  stockpile  of  Army  surface- 
to-air  missiles.  Furthermore,  the  results  of  the  atatletical  analyeis  were 
most  valuable  in  providing  several  clues  for  improving  ths  analysis  program. 

CONCLUSION.  In  conclusion,  It  should  be  emphasised  that  the  current 
program  for  the  analysie  of  date  derived  from  troop  training  tlights  ol  Nike 
Hercules  and  Hawk  missiles,  and  the  projected  improvements  in  the  program, 
will  not  necessarily  become  the  final  program  to  be  followed  for  these  or 
any  other  surface-to-air  missile  system*.  No  doubt  there  are  soma  factors 
due  to  aging,  design  improvements,  etc.  ,  which  hsve  not  yet  occurred 
and  which  will  affect  the  program.  All  of  these  factors  will  havs  to  be 
integrated  into  the  program.  Constantly  changing  data  end  improved  me.  hade 
of  analysis  make  it  necessary  that  the  analyel*  program  remain  flexible,  so 
that  the  maximum  amount  of  information  can  be  generated  from  the  basic 
input  d?t*i. 
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The  University  of  Wisconsin 

Statistical  decision  theory  nan  ha  applied  to  the  following  type  of 
problem,  Xp  . . . ,  Xm,  Yj, . . . ,  Yn  are  jointly  distributed  random  varlableu, 
whose  joint  distribution  la  not  completely  known,  but  la  known  to  be  one  of 
a  given  aet  of  possible  joint  distributions.  Alter  obaervlng  X , , . . . ,  Xm, 
but  before  observing  Yj, . . Yn,  somebody  has  to  make  a  deoision.  After 
the  decision  Is  made,  Yj, . . . ,  Yn  are  observed,  and  a  loss  Is  incurred  which 
depends  on  the  decision  chosen  and  on  the  observed  values  of 
Xj,,  . . . ,  Xm,  Yj, . . . ,  Y^ .  The  problem  Is  how  the  decision  is  to  be  baaed  on 

the  observed  values  of  Xj . Xm>  As  an  example  of  such  a  problem,  sup- 

1,  .ise  we  have  to  decide  how  Urge  a  feotory  to  build  to  produce  e  certain 
commodity.  The  X'a  might  bn  the  result*  of  e  survey  UKsn  to  examine  po¬ 
tential  demand,  and  tha  Y's  ere  the  demands  that  will  be  ubsuiv’H  In  the 
v«f  lout  ’'amounting  periods  after  the  factory  La  built, 

A  decision  rule  attaches  a  particular  rUolalon  to  each  possible  set 
of  X' s  that  might  bs  observed.  For  example,  In  the  oaie  of  the  factory, 
we  might  use  the  decision  rule  that  makes  weekly  oapaolty  equal  to  1000 
multiplied  by  the  sum  of  the  X's  .  More  generally,  a  decision  rule  may  use 
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randomization:  that  is,  it  aligns  a  set  of  probabilities  of  choosing  the  various 
decisions  to  each  possible  set  of  X's  that  might  ba  observed. 

We  Introduce  the  following  notation.  We  obbravlate  X, ,  . . . ,  Xm  by 
X,  and  Y| , . . . ,  Yn  by  Y.  We  use  the  symbol  0  as  an  index  for  the  possible 
joint  distributions  of  X,Y:  that  is,  a  particular  value  of  0  picks  out  a  parti¬ 
cular  distribution.  We  use  the  symbol  D  as  an  index  far  the  possible  decisions! 
a  particular  value  of  D  picks  out  a  particular  deolslon.  The  loss  we  incur 
when  X  is  observed,  D  is  chosen,  and  the  Y  is  observed,  is  denoted  by 
W(Y;D;X).  When  X,Y  are  discrete  variables,  f(x,y:fc>)  denotes  the  probability 
that  X  *  x  and  Y  ■  y  when  the  Joint  distribution  la  given  by  0.  When  X,Y 
are  continuous  variables,  f(x,y;0)  denotea  the  Joint  density  function.  A 
decision  rule  s  Is  defined  by  nonnegetlve  numbers  tt(DiX) ,  where  s(DiX) 
denotes  the  probebllity  assigned  by  the  decision  rule  s  io  choosing  the  tie- 
citron  D  when  X  Is  observed,  and  therefore  s(D|X)  »  1  for  each  A,  . 

We  denote  hy  i(f*j«)  the  oxpsotnd  value  of  the  lous  when  the  deolslon  ruin  ■ 
in  u»ed  and  the  true  joint  distribution  Is  given  by  0  .  Then  It  la  easily  see, 

that  when  X,Y  are  discrete  variables,  r(0;s)  “  njw,v  ,D;X)iViX,Y,0)«(D;X). 

X  Y  D 

When  X,Y  are  continuous,  we  leplaoe  summation  with  respect  to  X  nnr1  Y  by 


integration. 


Aw  a  numerical  Illustration  of  the  above,  suppose  we  have  to  decide 
whether  or  not  to  buy  a  nonguaranteed  device  for  $50U,  If  we  buy  the  device 
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end  It  fella,  we  mast  then  bay  e  guaranteed  device  for  (1000.  If  we  do. not  buy 
the  nonguaranteed  device,  we  buy  the  guaranteed  device  for  (1000.  The  propor¬ 
tion  of  defectives  turned  out  by  the  factory  whloh  produced  the  nonguaranteed 
device  La  unknown,  but  before  making  our  decision  we  can  obaerve  two  almllai 
devices  from  the  Lame  factory  to  see*  whether  or  not  each  failed.  We  Introduce 
the  following  notation.  Y  Is  doftned  to  be  i  if  tho  nonguaranteed  device  falls, 
and  0  otherwise.  X}  Is  defined  to  be  1  If  the  first  of  the  two  similar  devices 
to  be  observed  fall*,  and  0  otherwise,  with  a  similar  definition  for  X2  In  terms 
ot  the  second  device  to  be  observed.  ©  denotes  the  unknown  proportion  of  de¬ 
fectives  turned  out  by  the  factory  producing  the  nonguaranteed  devices.  Then 
X^Xj.Y  are  Independent  and  identically  distributed  random  variables,  each  with 
probability  ©  of  being  equal  to  1  and  probability  (1  -  ©)  of  being  equal  to  0. 

Wo  label  the  decision  to  buy  the  nonguaranteed  device  as  decision  number  1, 
and  the  decision  not  to  buy  it  as  decision  number  2.  Th<ru  the  loss  function 
La  seen  to  be  given  as  tullov.*.  W(''jliX)  ■  (500,  W(l;lfX)  •  $1500,  W(0)2|X)  « 
Wi'.lt2rX)  ■  (1000  no  matter  what  X  Is.  Then  If  the  decision  rule  sj  Is  given  by 
ijUiO.O)  •  1,  sidjO.D  -  ij(lil,  0)  ,  SjUil.l)  ■  0,  w*  find  that  rf©^)  «  SOU 

♦  llOO  ©  -  1000  02.  If  the  decision  mule  sj  If  given  by  s2(ll0,0)  •  0,  Sj(Ijo,1)  » 
s2(ltl,  9)  f  ^  i  a2(l}i,  l)i  •  U,rwe  find  that  *  UD00  -  400  ©  ♦  looo  ©2 . 

From  the  above,  It  is  clear  that  In  each  decision  problem  there  aru  In¬ 
finitely  many  deolslon  rules,  and  wo  need  some  way  of  oomparlng  the  goodness 


544 


Dsuign  of  Experiments 
of  one  decision  rula  to  another.  If  s^  and  s g  ere  two  decision  rules,  where 
i  r(0;»2)  for  all  possible  valuar  of  e  ,  and  r(®iBj)  <  r(«;s2)  for 
at  least  one  possible  value  of  8  than  wa  say  that  "s^  Is  bettor  then  Sg" 
and  wa  would  not  use  tha  decision  rula  Sg  .  For  example,  In  the  numerical 
illustration  of  tha  preceding  paragraph,  It  oan  be  verified  that  »j  is  batter 
than  s2  .  Wa  say  that  a  deal sion  rula  s  is  "inadmlesibla"  if  there  la  a 
better  daoision  rule  than  s  .  If  a  decision  rula  la  not  inedmlaalhla,  wo  call 
it  "admlsalbla" ,  How  can  wo  find  admiaaibla  decision  rules?  ror  simplicity, 
let  ns  limit  ouraelvaa  to  tha  onaa  where  there  la  a  finite  number  h  of  posaiblo 
joint  distributions  of  X,Y  so  that  wa  may  assume  that  the  poaalblo  values  of 
0  are  tha  Integers  from  1  to  h  inclusive.  If  b^, ....  bh  are  nonnegative 
quantities  adding  to  unity,  the  decision  rule  s  la  called  a  "Bayae  deolaion 

rula  relative  to  bi . b^  "  U  b^Uls)*. .  ,+bhr(his)  £  bjrUit)+. .  .-MyOtit/ 

for  u\ nry  decision  rule  t.  If  a  Is  oalled  simply  a  Bayes  decision  rule,* it 
means  that  there  U  some  set  of  nonnegativ«  quantities  bj , . . . ,  adding 
to  unity,  such  that  a  is  a  Bayes  decision  rule  relative  to  bi,.,,,b^.  a 
basic  theorem  states  that  every  admissible  daoision  rule  is  a  Bayes  dooieion 
rule .  Therefore  the  class  of  Bayes  decision  rules  contains  the  class  of  edr  -  i 
mlssible  decision  rules.  Since  it  is  computationally  aaoy  to  find  Bayas  de¬ 
cision  rules,  it  is  assy  to  find  admissible  decision  rules. 
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In  most  problsms  thers  are  Infinitely  many  admissible  decision  rules, 
only  one  ol  which  will  be  chosen  lor  use.  Whet  principles  can  be  used  for 
selecting  one  particular  decision  ruls  from  ell  the  admissible  decision  rules? 
There  Is  muoh  disagreement  about  this,  and  we  merely  mention  two  of  the 
various  principles  that  have  been  proposed.  One  principle  oalls  for  the 
assignment  of  "subjective  probabilities"  or  "degrees  of  belief"  to  the  various 
possible  values  of  B  ,  and  then  using  a  Bayes  daolaLn  rule  relative  to 
bj . bn,  where  b@  Is  set  aqual  to  the  subjective  probability  assigned 

to  the  value  B,  Another  principle  defines  M(s)  as  max  r(«i»),  and  then 

© 

proposes  using  a  decision  rule  which  minimises  M(s),  which  decision  rule 
Is  called  "mlnlmax". 


WHAT  TYPE  OF  STATISTICIANS  ARE  NEEDED  IN  RESEARCH 
AND  DEVELOPMENT  LABORATORIES 


Chairman:  Boyd  Harthbarger,  Department  of  Statistic*,  Virginia 
Polytechnic  Institute 

Panelists'.  Dr.  E,  L.  Cox,  Biometrical  Services  ARS,  Plant  Industry 
Station 

Dr.  Churchill  Eisenhart,  National  Bureau  of  Standards 

Dr.  Frank  E.  Crubba,  Army  Ballistics  Research  Laboratories 
Aberdeen,  Md. 

Mr.  John  L.  McDaniel.  Directorate  of  R  and  D,  Army  Missile 
Command 

Dr.  Paul  R.  Rider,  Cilice  of  Aero-Space  Re  search,  Wrigbt- 
Patterson  AFB 

Dr.  William  Wolman,  Goddard  Space  Flight  Center,  NASA 

Dr.  Donald  A,  Gardiner,  Mathematics  Division,  Oakridge 
National  Laboratories 


INTRODUCTORY  REMARKS  BY  THE  CHAIRMAN 
Boyd  Harshbarger 

Mathematical  statistics,  including  the  statistical  design  of  experi¬ 
ments,  is  a  new  discipline.  It  was  in  the  late  20'e  that  R.  A.  Fisher  began 
his  work  in  the  design  of  experiments,  In  1933  he  moved  to  the  University 
of  London  where  the  field  of  design  was  greatly  expended,  industry  wss 
■  tow  to  adopt  many  ul  the  ideas  developed  by  Fisher,  Yatss,  and  others. 

Dr.  Walter  She  whorl  of  the  Bell  Laboratories  might  be  considered 
the  "Father  ul  Statistic*  in  Industry1  .n  this  country,  but  it  was  ths  organi¬ 
zation  and  promotion  ability  of  General  Leslie  Simon  that  finaUv  caused 
the  proper  recognition  of  statistics.  Many  of  us  remenber  the  ten-du, 
v  .ndnr  courser  of  World  War  II  which  ware  directed  by  Dr.  Holorook 
Working  and  Dr.  Edward  Olds. 

Industry  and  government  installations  moved  quickly  from  qoa'lty 
control  to  the  design  of  experiments.  In  this  area  government  installations 
appear  to  have  developed  their  own  pattern  for  the  organisation  and  appli¬ 
cation  of  statistics.  On  the  other  hand,  companies  like  Ball  Laboratories 
developed  large  statistical  departments  where  coneulting  ie  concentrated  and 
where  statistical  research  Is  highly  developed.  Aa  an  example  of  an  Indus¬ 
trial  company  engaged  in  government  activities,  Hercules  Powder  Company 
has  been  active  in  the  application  of  statistics  to  propellant  research.  As 
early  as  World  War  II.  classes  were  set  up  at  the  Radford  Arsenal  and  soon 
after  full-time  i»t.atisticl«ns  were  employed. 
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Steel  companies,  like  U.  S,  Steel,  oil  companies,  chemical  industries, 
automobile  corporations,  and  most  of  the  defense  induetrire  row  employ 
statisticians.  Some  companies  in  certain  industries  have  been  more  success* 
ful  in  the  introduction  of  statistical  methods  than  others. 

it  is  fair  to  point  out  that  statistics  would  n't  hav*  developed  as  rapidly 
or  at  successfully  if  it  had  not  been  for  the  interests  of  government  agencies, 
in  this  connection  it  is  well  to  note  that  places  like  Aberdeen  Proving  Grounds 
have  been  most  successful  in  tue  use  of  statistics.  The  Army  Research 
Center  at  Durham  gave  aome  of  their  earlient  grant*  in  support  for  research 
in  the  design  of  experiments.  Redstone  Arsenal  and  White  Sands  Proving 
Grounds  have  used  statistical  methods  since  their  inception.  The  success 
of  these  two  installations  in  statistics  has  beon  limited  somew  lat  by  their 
ability  to  secure  competent  statisticians. 

What  1  have  aaid  about  the  Army  ia  true  of  other  military  installations, 
Including  the  Navy  and  Air  Force,  aa  well  a*  apace  program*. 

I  am  hoping  that  the  panel  will  outline  the  moat  efficient  method  for 
the  use  of  statistics  and  itetistlcal  design*.  I  am  suggesting  that  thsy 
outlins  the  best  method  of  approach  as  well  as  to  where  statistics  should 
be  located  In  an  organisation.  This  will  vary  from  organiaation  to  organi¬ 
zation  and  from  installation  to  installation.  I  do  feel,  however,  that  the 
panel  should  tell  u*  w.iat  they  are  doing  in  their  particular  activities  and 
what  they  might  think  would  be  beat  for  them  to  do, 

I  have  asked  each  of  th*  panelist*  to  give  us  a  ten  minute,  and  only 
tenmtrutea,  summary  of  their  *  uggsationa,  When  this  is  completed,  we 
will  then  throw  the  projostu  up *n  ior  discussion. 


STATISTICAL  PERSONNEL  ASSOCIATED  WITH 
AGRICULTURAL  RESEARCH 

Edwin  L.  Cox 

Statistician*  are  active  In  many  phases  of  the  work  of  the  U.  S.  De¬ 
partment  of  Agriculture.  The  remarks  which  follow  will  be  concerned 
with  statistic*  and  statistician*  principally  from  Statistical  Reporting 
Service  (SRS) ,  Forest  Service  (FS),  and  Agricultural  Research  Service, 
(ARS).  It  is  probably  correct  to  say  that  the  main  involvement  of 
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statistician*  with  R  and  D  problem*  will  be  located  in  the**  above  mentioned 
organisational  unit*. 

Before  commenting  ipeclfically  on  eome  of  the  difference*  in  need* 
and  practice*  a*  exemplified  by  theee  different  organisational  unit*  some 
overall  generalisations  will  be  presented,  The  statisticians  presently 
employed  will  be  found  in  the  main  to  have  had  education  and  training  In 
a  field  of  specialization  other  than  mathematics  or  statistics.  Of  necessity 
through  experience  or  further  training  this  prior  background  has  boon 
added  to  In  the  pursuit  of  professional  activity  In  statistics.  Tha  reason 
for  this  finding  is  not  entirely  due  to  shortages  of  graduates  with  statis¬ 
tical  majors.  It  has  bearing  on  various  other  factors;  historical,  personal, 
etc. 

The  Statistical  Reporting  Service  conducts  and  supervises  a  program 
of  Sample  Surveys.  Communication  with  tha  Census  Bureau  has  been  fre¬ 
quent.  The  Re aearch  and  Development  Branch  of  tha  Standards  and  Re¬ 
search  Division  has  a  program  of  Interpreting  and  developing  tha  practice 
of  sampling  methods  for  production  and  other  estimates.  3RS  has  a  definite 
program  of  sending  statistical  personnel  where  they  can  pursue  a  program 
of  graduate  training  in  statistics.  As  might  be  expected  from  the  mission 
of  the  organisation,  the  training  received  is  mainly  in  theory  and  methodology 
of  sampling  *urv*y»  and  censuses. 

The  Forest  Service  has  placed  or  intend*  to  place  statistical  personnel 
at  all  regional  laboratores.  The  policy  has  generally  been  to  expect  tnat 
r>r.h  statistical  personnel  have  training  in  forestry,  Additional  training  In 
eUtlatiua  has  In  the  past  been  accomplished  in  part  by  the  provision  of 
special  courses  by  'orcct  service  statisticians,  The  statistical  contribution* 
from  the  Forest  Service  have  been  typically  in  the  areas  of  technique  and 
methodology.  Rather  sper.ial.taed  techniques  of  sampling  have  received 
statistical  evaluation. 

In  the  Agricultural  Research  Service  most  of  the  statistical  per¬ 
sonnel  are  organisationally  located  in  a  unit  called  Biometrical  Services. 
Except  for  those  assigned  to  the  four  utilisation  laboratories,  the  m-mbers 
oi  Biometrical  Services  are  stationed  in  the  Washington,  D.  C.  area. 

Their  statistical  consulting  service  to  field  stations  Is  provided  by 
correspondence.  In  addition  to  statistical  aid  to  ARS  scientists  In  Washington 
and  BeltsviUe  and  in  the  field  a  computing  laboratory  it  maintained  as  an 
integral  part  of  the  unit.  Biometrical  eervices  personnel  In  the  main  have 
not  received  their  primary  graduate  training  in  statistic*.  Some  have  pursued 
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additions!  study  in  statistics  or  h»vs  gained  competence  by  experience. 

The  scope  of  the  subject  matter  which  may  come  under  consideration  with 
respect  to  problems  of  design  and  analysis  of  experiments  cover*  the  com* 
plate  range  of  agricultural  experimentation,  Statloticai  contributions  from 
this  group  havo  come  principally  in  association  with  subject  matter  special* 
isti,  Several  bulletins  of  specially  statistical  interest  have  been  produced. 

It  would  seem  from  the  experience  of  these  working  organisations  that 
a  firm  acquaintance  with  subject  matter  is  important  for  statistical  per¬ 
sonnel  associated  with  agricultural  research.  A  coordination  of  statistical 
techniques  with  this  foundation  seams  to  produce  a  person  who  has  rapport 
with  ths  scientists  and  can  aid  Mid  advise  them  on  numa  rical  problems, 
it  maybe  that  some  areas  dessrving  statistical  tools  hav?  been  ignored  but 
until  there  is  a  felt  need  for  attention,  these  areas  are  likely  to  remain  un- 
inveetlgated. 


THE  NEEDS  OF  THE  STATISTICAL  ENGINEERING  PROGRAMS 
OF  THE  NATIONAL  BUREAU  OF  STANDARDS 

Churchill  Eisenhart 

•  welcome  this  opportunity  to  review  the  staffing  needs  and  recruit¬ 
ment  difficulties  of  the  statistical  engineering  program*  of  the  National 
Bureau  of  Standards  because  1  feel  that  they  are  symptomatic  of  the  needs 
and  recruitment  problems  of  many  of  the  statistical  advisory  and  consulting 
group*  in  Government  laboratories,  To  provide  the  setting  for  my  story, 
let  tiuj  begin  with  a  few  words  about  the  programs  and  function*  cl  thu 
three  statistical  group*  at  the  National  Bureau  of  Standards, 

The  major  statistical  program  of  the  National  Bureau  of  Stand»»4* 
la  that  of  the  Statistical  Engineering  Laboratory  of  the  Applied  Mathe¬ 
matic*  Division,  c:.  the  main  "campus"  of  the  Bureau,  ir  Washington, 
n.  C,  This  group,  of  which  I  was  the  Chief  from  its  formal  establishment 
on  July  1,  1947  until  June  10,  1963,  is  now  under  the  direction  of  Joseph 
M,  Cameron,  Chief,  andJonnR,  Rosenblatt,  Assistant  Chief.  The  prin¬ 
cipal  function  of  the  Statistical  Engineering  Laboratory  is  to  serve  in  an 
advisory  and  supporting  capacity  to  th*  Bureau's  scientific  and  technical 
personnel  on  the  application  of  modern  probability  and  statistical  methods 
in  the  physical  sciences  and  engineering.  Its  staff  also  conducts  research 
on  the  theory  and  techniques  of  statistical  inference  and  statistical  design 
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of  experiment!,  with  ipecial  reference  to  problems  that  arise  ia  physical 
science  and  engineering  experimentation:  and  to  a  lesser  extent,  on  math*- 
matical  end  statistical  aspects  of  the  definition,  measurement,  and  speci¬ 
fication  of  the  reliability  of  components  and  asoumblies,  with  particular 
attention  to  the  evaluation  of  the  reliability  of  electronic  component*  and 
eyetems. 

There  are  two  other,  much  (mailer,  statistical  g’  oupa  at  the  National 
Bureau  of  Standarde,  Ore  of  these  is  loca'ed  In  the  Polymers  Division,  on 
the  Washington  campus;  the  other,  in  the  Office  of  the  Manager  of  the  Boulder 
Laboratories  of  the  National  Bureau  of  Standards,  in  Boulder,  Colurado, 

Both  of  these  groups  operate  independently  of  the  Statistical  Engineering 
Laboratory,  The  Polymers  Division  group  concentrates  on  development 
of  statistical  techniques  for  the  design,  analysis,  and  interpretation  of 
physical  and  chemical  data  pertinent  to  the  evaluation  of  the  measurement 
techniques  in  the  field  of  polymers)  and,  in  recent  years,  has  been  actively 
engaged  in  etudiee  of  measurement  processes  dealing  with  polymeric  materials 
from  the  viewpoints  of  Interlaboratory  standardisation  and  tire  noed  tor 
standard  ruftrenca  materials,  The  Boulder  Laboratories  group  specialises 
in  adapting  existing  general  statistical  methods  to  the  experimental  work 
of  the  Boulder  Laboratories,  and  to  developing  mathematical  models  of 
■  tochantic  phenomena  of  lntereat  in  scientific  and  engineering  programs 
.,i  Ihc  Boulder  Laboratories, 

Both  the  service  and  the  research  activities  of  til  three  of  these  groups 
would  benefit  from  additional  personnel.  1  shall,  however,  restrict  my 
comments  principally  to  ths  needs  of  the  Statistical  Engineering  Laboratory, 
as  this  is  the  group  wth  whL.I-  I  am  most  familiar. 

To  begin  with,  the  Statisti-il  Engineering  Laboratory'*  service 
acridities  -*  its  provision  of  advice  and  assistance  to  NBS  staff  nie.vAn'i 
on  statistical  awpect*  of  their  calibrations,  research,  and  development 
programs  --  is  continually  hampered  by  shortage  of  personnel,  Thu 
available  "consultant*"  are  frequently  involved  in  four  or  five  major 
consulting  projects  at  onv  time,  and  therefore  forced  to  reduce  the  time 
spent  on  any  one  project.  The  Laboratory  has  continual  need  for  at'LHonal 
specialists  in  experimental  statistic*  to  enabls  it  to  give  somu  projects  the 
detailed  thorough  study  they  warrant.  The  background  of  these  specialists 
should  Include  a*  a  minimum  a  strong  M5  in  statistics  with  some  first¬ 
hand  experience  in  the  design  and  analysis  of  experiments.  Second,  both 
tho  service  and  the  research  actlvitiee  of  the  Statistical  Engineering 
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Laboratory  would  benofit  if  th*  Laboratory  ware  able  to  attract  a  auccaation 
of  topnotch  young  Ph,  D'a  in  mathematical  atatiatlca  who  daalr*  to  gain  tom* 
firathand  contact  with  application*  in  the  phyalcal  aciencaa  and  engineering 
before  devoting  the  remainder  of  their  career*  to  teaching  in  a  unlvaralty 
or  college,  or  to  working  in  induatry  or  in  one  of  the  ever-increaelng  number 
of  non-profit  reaearch  tnatltutea.  The  aciantific  and  technical  program*  of 
the  National  Bureau  of  Standard*,  embracing  almost  every  area  of  th*  phyal¬ 
cal  science*,  afford  unusual  opportunities  for  gaining  experience  in  the  appli¬ 
cation  of  modern  statistical  tools, 

Unfortunately,  both  from  the  Bureau's  viewpoint  and  from  the  long- 
range  view  oi  application  of  statistical  methods  in  the  physical  sciences 
and  engineering  in  this  country,  the  number  of  qualified  applicants  for 
positions  in  the  Statistical  Engineering  Laboratory  at  all  levels  is  almost 
nil.  There  are  several  reasons  for  this  situation,  Let  ue  consider  first 
thu  Laboratory's  "rad  carpet"  positions,  its  Postdoctoral  Resident  Re¬ 
search  Assoclat* ships  in  probability  and  mathamatlcal  statistics,* 

A  recipient  of  one  of  thsi;*  Research  Associateships  is  expected  to 
devote  his  efforts  entirely  to  advanced  training  and  reaaarch  in  some 
aspect  of  probability  and  mathematical  statistics  related  to  the  work  of 
the  Statistical  Engineering  Laboratory.  In  recent  years,  research  activities 
of  the  Laboratory  have  been  directed  toward  the  evaluation  of  exietlng  and 
thu  development  of  new  statistical  techniques  of  value  for  the  applications 
In  physical  science  and  engineering  experimentation,  with  special  attention 
to  "practical  properties"  of  statistical  techniques,  including  questions  of 
power,  efficiency,  robustness,  consaquenos*  of  misapplications.  Th* 
"advanced  training"  may  consist  merely  of  passively  observing  nature, 
and  t;»o  manner  in  which  we  c&r«y  out,  our  concultativ*  and  advisory  ser¬ 
vicer  in  probability  «ud  mathem  u  id  cal  statistic  *  The  Research  Asso¬ 
ciate  may  elect  to  participate  in  one  of  these  consultative  and  advisory 


These  awards  are  part  of  a  broad  program  eponsored  by  tha  National 
Bureau  of  Standards,  In  cooperation  with  th*  National  Academy  of 
Sciences  --National  Research  Council.  Their  purpose  is  "to  provide 
younu  scientists  of  unusual  ability  and  promise  an  opportunity  for  fui.du- 
mental  research  in  various  branches  of  th*  physical  and  mathematical 
sciences".  Intended  principally  for  recent  Fh.  D's  and  Sc.  D's,  tha  awarda 
carry  an  annual  gross  stipend  equal  to  th*  OS- 12  entrance  salary  in  th* 

U,  S.  Civil  Service.  Applicant*'  qualifications  are  evaluated  by  aboard 
of  selection  appointed  by  th*  National  Academy  of  Sciences  --  National 
Research  Council. 
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activities  to  gain  insight,  inspiration,  and  some  practical  experience  -  -  but 
he  is  under  no  obligation  to  do  so.  tn  spite  of  the  honor  accruing  to  the  re¬ 
cipient*  of  these  Research  Associateships  by  virtue  of  the  administration 
of  tlie  program  by  the  National  Academy  of  Sciences  --  National  Research 
Council,  and  of  the  definitely  preferential  status  accordbd  them  in  the 
Statistical  Engineering  Laboratory,  in  4  out  of  the  7  yeare  of  the  program's 
existence  there  were  no  applicants  tor  the  Research  Assoclateship  in  prob¬ 
ability  and  mathematical  statistics  in  the  Statistical  Engineering  Laboratory, 
the  present  year  being  one  of  the  "dry"  years,  and  in  only  on*  year  ware 
there  as  many  as  2  candidates.  In  contrast,  each  year  there  have  been 
ample  applicants  in  "pure"  mathematics,  and  in  many  branches  of  physic* 
and  chemistry, 

There  are,  I  believe,  at  least  four  reasons  for  this  dearth  of  appll- 
canta  (or  these  Postdoctoral  Resident  Research  Associate  ship*  in  prob¬ 
ability  and  mathematical  statistics!  First,  there  are  far  fewer  Ph.  D's 
specialising  in  probability  and  mathematical  statistics  than  in  the  other 
branches  of  mathematics  --  and,  a  fortiori,  than  in  the  more  popular 
branches  of  physics  and  chsmlstry,  Sscond,  although  the  Statistical 
Engineering  Laboratory  of  the  National  Bureau  of  Standards  is  on*  of 
the  principal  centers  of  research  In  the  application  of  statistical  theory 
and  methods  in  the  physical  and  englnssrlng  science*  in  this  country,  Its 
activities  are  primarily  of  Interest  to  groups  engagtd  in  measursment, 
calibration,  and  testing  in  the  physical  and  engineering  sciences,  and  are 
of  very  little  Interest  to  --  and  carry  no  special  authority  in  --  the  de¬ 
partments  of  mathsmatlcs,  where  the  majority  of  courses  in  mathematical 
statistics  are  taught.  In  contract,  the  work  of  many  dl visions  of  the 
Na„»onal  Bureau  of  Standards  is  vital  to  research  and  development  programs 
in  the  physical-science  «nd  engineering  departments  nt  «oU*gsa  and 
universities.  Third,  the  present-day  training  in  statistics  at  most  of 
the  universities  or  colleges  In  this  country  is  so  strongly  oriented  toward 
theory,  with  little  attention  to  applications,  that  the  majority  of  the 
ev.cceesful  candidates  for  MA  and  PhD  degrees  in  probability  end  mathe¬ 
matical  statistics  have  already  developed  a  strong  distaste  for  application*, 
partly  (rum  the  expressed  contempt  of  "mathematician's  mathematicians" 
towards  anything  "applied"  in  contrast  to  "pure",  and  partly  from  lank 
exporience  with  applications  and  resulting  fear  of  the  unknown.  Consequently, 
even  explicitly  research-oriented  "positions"  in  a  government  laboratory 
are  ahied  away  from,  being  regarded  as  having  an  applied  "taint".  Fourth, 
the  email  number  of  individuals  who  do  have  the  requisite  training  and  interest 
are,  for  the  most  part,  sither  lured  into  industrial  applications  by  the  much 
higher  ealarles  ottered  by  industry  and  ths  ever-growing  number  of  "non- 
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profit"  private  research  institutes,  or  take  poeitions  with  universities 
where  they  ..an  engage  simultaneously  in  teaching  and  research  umler 
the  contract  arrangements  now  generally  available,  for  the  most  part 
with  Government  funds.  These  latter  are  able  to  offer  the  usual  induce¬ 
ments  of  university  appointments  and  titles,  less  ovringent  working  hours, 
opportunity  for  teaching  and  private  consulting,  for  extra  mu  tie  y  from  the 
"outside" ,  and  now  also  in  many  cases  equal  or  higher  salaries  at  well, 

Let  us  now  consider  the  situation  at  the  lower  professional  levels  in 
the  Statistical  Engineering  Laboratory.  In  comparison  with  other  section! 
of  the  NBS,  and  in  relation  to  customary  management  norms,  the  Statis¬ 
tical  Engineering  Laboratory  has  always  been  "top  heavy"  ,  that  is,  has 
always  had  an  unusually  large  fraction  of  Its  staff  in  the  upper  professional 
brackets.  This,  1  believe,  le  characteristic  of  most  consulting  groups, 

In  the  1950's  ("the  good  old  days")  when  our  recruiting  at  the  PhD-levul 
was  more  euccessful,  the  Laboratory  wee  especially  top-heavy  with  PhD's) 
and  the  median  Civil  Service  grade  ot  the  11  professional  members  of  the 
year-around  staff  was  GS-12,  i.e.  ,  one  grade  above  the  entrance  level  for 
brand-new  PhD's  in  Mathematics  and  Mathematical  Statistics,  Today 
(October  1963)  the  corresponding  median  grade  of  the  year-round  staff,  in¬ 
cluding  myself  es  an  "honorary  member",  is  OS-13;  i.e.  one  grade  above 
the  entrance  level  for  "quality"  PhD's  in  mathematics  and  mathematical 
statistics  --  and  only  "quality"  PhD's  seem  to  be  avallablo. 

The  Statistical  Engineering  Laboratory  has  always  managed  to  get 
along  with  far  fewer  personnel  in  the  lower  professional  levels  than  most 
statistical  consulting  and  advisory  groups  because  it  has  never  needed  a 
*Ucc>Le  computing  group  within  the  Laboratory  itself  on  account  of  the 
existence  and  general  avallablH*y  within  the  Bureau  of  the  computing  ser¬ 
vices  of  the  lNBS  Computation  Laboratory.  The  senior  staff  of  the  Statisti¬ 
cal  Engineering  Laboretory,  on  Li,e  other  hand,  have  been  obliged  for  many 
year*  to  do  for  themselves  various  " chores''  that  normally  would  u,  the 
assigned  work  of  "graduate  assistants"  at  a  large  university  and  of  GS-S  ana 
GS-7  level  professional  assistants  in  many  other  laboratories.  Our  short¬ 
age  of  support  staff  at  these  levels  has  stemmed  from  the  almost  complete 
lack  of  qualified  candidates  for  lower-and-middlo  -auk  professional  positions 
in  analytical  and  mathematical  atatistlce,  a  situation  which,  in  turn,  i,imi 
from  the  fact  that  in  the  United  States  statistics  la  largely  a  post-graduate 
field  of  specialisation.  As  a  result  there  are  very  few  applicants  for  Statis¬ 
tician  positions  at  the  bachelor' e-degree  (GS-5  or  GS-7)  entrance  levels  who 
have  training  In  statistics  that  is  comparable  in  scope  and  depth  to  the  train¬ 
ing  in  physics  or  chemistry  that  is  required  of  applicants  for  Physicist  and 
Chemist  positions  at  the  same  entrance  levels. 
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In  other  (action*  of  the  Bureau,  where  the  work  involve*  the 
traditional  field*  of  chemistry,  physic*,  and  engineering,  there  l* 
a  continual  influx  of  "new  young  blood"  directly  out  of  college*  and 
universities,  at  the  bachelor'*  and  master's  degree  level*.  These 
junior  professional*  assist  the  higher-ups  with  the  more  straight¬ 
forward  phases  of  the  day-to-day  operations  of  the  groups  concerned, 
and  a  few,  througn  on-the-job-experienca  and  in-service  training,  rise 
in  due  course  to  higher  positions  In  the  organization.  To  date,  the 
Statistical  Engineering  Laboratory  has  been  obliged  to  get  along  with¬ 
out  the  "normal"  complement  of  lower  level  aeeistants  on  account  of 
the  peculiar  circumstance  a  of  the  statistical  profession,  It  may  be, 
however,  that  in  time  the  current  trends  in  the  undergraduate  mathe¬ 
matics  curriculum  will  produce  an  increased  number  of  qualified 
npplicante  for  junior  positions. 

The  staff  of  a  consulting  group  llko  the  Statletical  Engineering 
Laboratory  will,  nevertheleee,  alwaye  tend  to  be  top-heavy  bscause 
of  the  nature  of  lie  work,  A  statistical  consultant  has  to  be  prepared 
to  handle  a  wide  variety  of  problems:  when  a  physicist  telsphones  to 
say  he  hae  a  problem  in  loeit-aquaree  anelysla,  It  may  turn  out  that 
hie  real  problem  (from  tbs  professional  statistician's  point  of  vlsw) 
is  in  experiment  design,  or  numerical  analysis,  or  components  of 
variance.  The  consultant  who  ii  assigned  to  provide  advice  must 
have  a  sufficiently  broad  background  to  identify  the  problem  correctly 
and  to  be  able  to  apply  the  relevant  theory  and  methods.  An  especially 
important  part  of  the  requisite  background  in  tho  the  rough  advanced 
training  in  probability  and  distribution  theory  that  enables  the  statis- 
iclan  to  steer  a  safe  course  through  the  adaptation  of  standard  tech¬ 
niques  for  application  in  ine  almost  universally  non-standard  circum¬ 
stances  that  arise  in  physical  science  >  .vestlgation*. 

While  it  ie  not  necessery  that  each  consultant  be  an  expert  In  evvi/ 
field  of  statistics  (this  would  be  abeurd),  it  Is  also  true  that  consulting 
work  will  not  suit  the  taetes  of  a  narrow  specialist  who  doesn't  cars 
to  know  about  any  field  except  hie  own. 

In  eummary,  our  efforts  to  recruit  brand-  new  college  graduatea 
MA'i,  and  PhD's  has  been  discouragingly  unsuccessful  In  recent 
year*.  1  have  summarized  some  of  the  factors  contributing  tc  this 
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situation.  Fortunately  we  have  beer,  a  bit  more  auccessful  In  recruiting 
MA1  e  and  PhD*  a  who  have  been  "out*  for  a  year  nr  two;  and  eome  of  our 
MA1*  have  grown  into  full  PhD1*  right  on  the  job,  through  on-the-job 
experience  and  "outside"  job-related  couree  work;  and  at  leaet  two  of 
our  preeent  etaff  are  headed  in  the  earn*  direction.  We  juet  with  that 
more  promlalng  young  men  and  women  would  "give  u*  a  try"  enroute 
to  their  ultimate  career*. 


SOME  THINGS  THE  STATISTICIAN  SHOULD  KNOW 
Frank  E,  Grubb* 


In  continuing  thi*  completely  unreh*ar»ed  program,  perhaps  th*  first 
remark  that  should  be  made  concerning  what  type*  of  statistician*  are 
needed  in  R  it  O  laboratories  is  that  they  should  be  good  statistician*! 

The  next  remark  is  that  the  statistician  should  know  quite  well  or  learn 
an  thoroughly  as  possible  th*  field  of  application.  This  is  especially 
durable  or  necessary  in  ths  engineering  and  scientific  fields,  or  other 
fields,  where  physical,  mathematical,  biological,  etc.  models  are  in 
many  cases  already  available.  To  speak  only  in  statistical  term*  is  s 
lost  cause  Indeed!  Also,  the  statistician  in  many  R  b  D  laboratories 
is  a  member  of  a  team  -•  his  methods  are  well  worthwhile  and  needed, 
although  it  seems  to  be  a  rather  natural  fact  that  the  statistician  is  ms.'e 
prone  to  criticism  than  any  of  lus  colleagues.  But  this  is  ah  in  the  game 
of  things,  and  we  must  remember  that  the  non- statistical  Investigator  is 
often  a  bit  jealous  of  the  power  and  usefulness  of  statistical  method''  In 
Homn  problems,  on  the  other  hand,  statistics  cannot  help. 

The  types  of  statisticians  needed  depend  very  much  on  what  they  actu¬ 
ally  do  in  R  tt  D  laboratories,  so  we  must  examine  this  momentarily.  There 
is  the  obvious  need  for  occasional  statistical  analysis  of  data  (includina  ‘he 
common  tests  of  significance,  regression  analysis,  etc.)  and  the  statistical 
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design  and  analysis  of  scientific  experiments  .  Factorial  experimentation, 
incomplete  block  designs,  Latin  and  Graeco*' Latin  Squares,  lattices,  etc. 
are  all  of  considerable  importance,  as  is  also  the  search  for  optimum 
operating  conditions  utilising  Box  techniques.  Interpretations  of  the  data, 
however,  must  always  bs  in  terms  of  the  physical  picture  involved,  as 
this  Is  the  baaic  requirement  originally  desired. 

At  the  Ballistic  Research  Laboratories  (  and  no  doubt  R  V  D  labora¬ 
tories  generally),  curve  fitting  111  of  importance.  1  distinguish  between 
the  fitting  of  "physical"  laws  or  curves  in  genera)  and  the  fitting  of  fre¬ 
quency  distributions,  In  ordinary  curve  fitting,  the  phya  icist  or  engin¬ 
eer  is  oftentimes  trying  to  linearise  his  data  in  addition  to  trying  to  got 
the  "best"  fit.  He  plots  data  on  a  logarithmic  scale,  etc,  The  statii  - 
tician  could  be  especially  helpful  (If  he  can  get  hit  hands  on  the  data!) 
since  he  Is  usually  adept  in  the  use  of  transformations  ,  least  squares 
and  testing  the  significance  of  parameters.  (Iteration  is  often  required  to 
estimate  parameters, ) 

There  is  also  the  need  for  fitting  frequency  distributions  .  The 
Pearson  system  of  frequency  curves  is  quits  general  (including  even 
the  Waibull  distribution  of  much  current  Interest)  end  the  method  of 
moments  very  useful  in  practice  ss  compared  to  maximum  likelihood 
estimation,  which  although  theoretically  elegant  and  efficient,  is  often  not. 
■.«j  satiety ing  practically,  (i  must  remark  that  stalls  Helens  coming  out 
cf  the  graduate  schools  now  have  most  likely  not  studied  the  Pearson 
system  of  frequency  curves  or  the  Gram-Charlier  Scales,  etc,  I) 

The  big,  new  field  of  endeavor  in  Army  R  It  D  work  la  that  of 
Operations  Research.  *"d  »n  this  I  include  the  very  important  field  (if 
Weapon  Systems  evaluation.  In  the  latter  case,  as  in  much  OR  work, 
we  often  get  somewhat  away  from  significance  testing,  experimental 
designs,  etc.  and  come  face-to-faca  with  the  requirement  for  probabi- 
lis  tic  models.  We  predominantly  speak  of  "hit  probabilities,  "  "kill 
probabilities,  "  coverage  problems  concerning  the  overlap  of  weapon 
"lethal  areas"  on  the  target  area,  dletributlon  of  the  range  of  engage¬ 
ment,  chance  of  winning  an  engagement,  etc,  But  again,  the  statis¬ 
tician  is  badly  needed.  The  calculation  of  many  probabilities  of  hitting, 
for  example,  is  closely  tied  in  with  the  distribution  of  quadratic  forms 
in  normal  variables,  the  distribution  of  range  of  engagement  If  often 
of  the  Pearson  Type  HI  or  Gemma  class  of  frequency  distribution,  etc. 
Also,  for  weapon  systems  evaluation  problems,  there  is  ever-present 
the  requirement  for  combining  several  frequency  distribution*  to 
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obtain  the  single  or  overall  eharactenaation  of  weapon  system  perform¬ 
ance,  if  possible,  Reliability  studies  represent  a  very  important  facet 
of  the  field  of  weapons  evaluation  and  the  statistician  can  be  very  valuable 
here  too,  Some  reliability  theory  should  be  In  the  present  curricula  for 
statistics,  Thus,  the  properly  qualified  statistician  can  do  m-  vi  for  this 
new  field  and  he  is  a  necessary  part  of  it, 

Finally,  my  time  ie  getting  short,  but  1  must  mention  that  a  vary 
special,  highly  qualified  type  of  statistician  is  very  often  needed,  He  ie 
the  type  that  must  bs  right  up-to-date  with  the  mast  recent  advances  in 
the  state  of  the  art  theoretically,  He  is  the  one  who  is  pushing  the 
frontiers  forward  in  many  areas,  combining  developments  from  many 
areas  and,  for  example,  including  tims  series  studies,  variance -component 
analyses,  multi- variate  normal  theory,  the  treatment  of  errors  of 
measurement,  precision  and  accuracy,  ate.  The  practical  types  of 
problems  I  speak  of  are  « 'ready  with  us  and  the  demand  it  most  urgent 
for  accurate  estimation  procedures ,  1  refer  to  the  tracking  data 
analysis  problem  for  missile  trajectories,  which  Dr,  David  Duncan 
will  discuss  at  tha  general  esse  ion  tomorrow.  For  such  problems,  we 
see  the  need  for  the  most  highly  trained  etatlstlcian  and  tha  requlrament 
is  likely  to  grow  considerably. 

r  remark  and  observe  that  many  of  the  papers  presented  in  this  con¬ 
ference  indicate  the  types  of  statisticians  needed  in  R  k  D  laboratories, 

in  Nummary,  thsrefore,  In  R  it  D  laboratories  wt  need  both  the 
applied  statistician  who  will  analyae  data  statistically,  carry  out  design 
of  experiments,  stc,  and  help  oil. ore  do  their  job  better,  Alt''  we  need 
the  theoretical  statistician  for  many  very  oomplox  problems  cropping  up 
on  the  frontiers  of  knowledge,  in  either  case,  however,  the  statistician 
must  know  the  physical  or  engineering  fields  quits  well  in  order  that  h,. 
may  rrmke  the  bes  t  contribution  pos  •  ible. 
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DESIRABLE  ATTRIBUTES  OF  STATISTICIANS 
IN  R  it  D  LABORATORIES 

John  L,  McDaniel 

The  word  "laboratory"  in  the  Defenea  context  has  many  meaninga. 
Used  looaely,  as  we  all  tend  to  do,  it  encompasses  the  full  spectrum  of 
weapon  technology  from  basic  research  through  production.  Further, 
each  service  hat  problems  unique  to  their  mite  ions,  and  thus  have 
discreet  requirements  for  itt  laboratories.  Naturally,  my  remarks 
will  be  signed  toward  our  Army  laboratories  In  general  and  our  mlksile 
laboratories  of  the  Army  Missile  Command  !n  particular.  I  believe 
however  my  remarks  will  be  sufficiently  general  to  be  useful  in  our 
broad  dlec«ss  Ion  of  Ret  earch  k  Development  Laboratoriea. 

The  R  k  D  laboratories  of  the  Army  Missile  Command  ars  an 
organic  part  of  a  Commodity  Command.  The  commodity  Is  mis  slles, 
therefor*  It  Is  an  intrinsic  responsibility  of  our  laboratories  to  concern 
themselves  with  a  mis  sion  which  covers  first,  research)  second,  the 
generation  of  new  missile  concept*  to  fulfill  the  requirements  of  the 
future  Armyi  and  thtr,'  th*  technical  support  required  In  the  fielding  of 
missile  eye  terns  now  In  devslopment.  In  aarrying  out  this  broad 
mission  the  laboratories  must  be  constantly  mindful  of  the  system 
characteristics  which  assure  that  the  weapon  will  function  reliably  in 
the  environment  anywhere  in  ihe  world  that  the  Army  is  required  to 
fight, 

The  formula  for  discharging  this  responsibility  contains  many 
terms.  One  of  the  t«*»«*  is  men  and  It  is  this  term  we  will  dis¬ 
cuss,  Th*  c«r.c*itful  operation  of  R  k  D  laboratories  requires  teamwork 
of  the  men.  Th*  team  is  made  up  of  persons  trained  In  the  physical 
sciences  and  in  the  various  fields  of  engineering,  The  etatlsticia,,  l.- 
. i,-iw  a  major,  rather  than  a  casual  partner  of  this  team. 

My  first  desirable  attribute  then  for  ths  statletlclanfor  R  k  D 
laboratories  is  an  individual  to  work  as  a  team  member  in  provls  lonh.a 
reliable  weapon  t  ys  terns.  To  operate  effectively  as  a  member  of  this 
team  the  statistician  must  have  a  fair  amount  of  formal  education  in 
his  field  and  a  great  amount  of  appreciation  for  the  physical  sciences 
and  for  engineering. 
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Tlio  statistician  plays  a  key  role  in  the  area  of  reliability.  Reliability 
is  a  function  ox  the  Army's  decision  from  beginning  to  end.  Men  experienced 
in  this  unique  business  can  look  objectively  at  a  development  program,  almost 
before  it  begins,  and  Identify  the  reliability  problems  we  can  expect  to  encounter. 
Reliability  must  bo  built,  into  all  the  decisions  which  are  made  through  an  en¬ 
tire  program,  through  deelgn,  teste  of  successive  stage*  of  design,  through 
corrections  made  during  design,  through  further  testing,  through  production 
and  the  aseurance  of  quality  production,  through  supply  and  protection  of 
quality  in  the  supply  network. 

My  second  desirabl n  characteristic  of  a  statistician  in  the  R  ii  D  labora¬ 
tories  then  is  an  individual  trained  in  the  techniques  of  reliability  as  this 
reliability  relates  to  a  weapon  system  throughout  its  life  cycle, 

Decisions  regarding  the  missile  systems  generated  by  the  R  it  D  labora- 
torteo  must  be  made  by  men  who  have  to  stand-up  and  be  counted.  Stand 
up  and  be  counted  by  the  men  who  must  answer  to  the  combat  soldier  who 
takes  the  missile  system  into  the  battlefield  with  his  Ufa  in  his  hands.  We 
cannot  have  those  decisions  made  by  other  people  for  us.  These  decisions 
must  be  made  on  the  basis  of  our  knowledgs,  reinforced  by  the  knowledge 
of  others  In  equally  objective  positions.  In  dealing  with  the  competitive 
commercial  world  w*  must  be  in  a  position  truly  to  control  our  missile 
business,  We  must  have  technically  competent  people  in  our  laboratories 
who  have  i.o  conflicting  interests  and  who  therefore  cart  afford  to  be  objective, 

Third,  then  the  statistician  in  the  R  fc  D  laboratories  m ust  ba  technical1  y 
compete  nt  In  mi  sill*  technology  and  must  have  a  full  measure  of  objectivity , 

There  are  those  who  ballevw  reliable  missile  systems  <-c.n  be  pro¬ 
vided  to  the  user  without  governnis.oi  competence  in  in-house  RID  lab¬ 
oratories.  This  belief  ignores  tomr  very  basic  principles. 

First,  the  competitive  commercial  induatry  engaged  In  missile  system 
engineering  require*  control.  To  apply  tht*  control  a  capability  to  con¬ 
trol  la  required,  not  just  by  suggestion  but  by  direction  when  necessary, 

Sen  mu,  the  capability  to  control  requires  knowledge  and  experience. 

Without  this  experience  one  must  rely  on  faith  and  hopeful  trust,  The 
engineer  cannot  know  whether  his  faith  is  well  placed  unlee*  ho  has  first 
hand  knowle  dge. 
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Engineering  end  scientific  knowledge  deteriorate*  in  a  vacuum. 

Unless  the  engineer  ha*  the  responsibility  for  actual  hardware  work,  for 
getting  his  hands  dirty,  he  will  become  the  kind  of  engineer  who  looks  over 
drawings  and  hopes  he  is  right,  but  has  a  measure  of  doubt  •*  to  his  wis  • 
dom  since  he  i*  not  current  with  the  state-of-the-art. 

My  fourth  premise  then  is  that  the  »tatlstlclan  in  the  R  It  TS  laboratories 
must  have  confidence  in  his  own  knowledge.  This  knowledge  must  come  from 
working  with  the  hardware,  not  just  sitting  in  a  staff  position  and  waiting 
for  a  problem  to  be  brought  to  him, 

In  order  to  afford  to  the  scientists  and  engineers  the  requl:  ed  know!*  rig* 
a  small  amount  of  our  business  must  be  done  in-house.  Enough  to  allow  us 
to  control  the  rest  of  our  missile  business.  I  have  heard  that  a  good  many 
years  ago  when  Henry  Ford  started  to  build  a  small  steel  plant  a  committee 
of  the  Congress  called  him  and  questioned  him  about  monoply.  "Isn't  there 
enough  steel  in  the  country  for  you",  he  was  asked,  "yes  Sir"  Ford 
answered,  "well  then",  the  committee  asked,  "why  do  you  build  a  steel 
plant?"  Henry  Ford's  answer  Is  naid  to  have  been  -*  "Gentlemen,  to 
learn  enough  about  making  steel  to  know  whet  I'm  buying  from  the  steel¬ 
makers.  " 

Gentlemen,  I  submit  finally  we  require  statisticians  in  the  R  h  D 
laboratories  as  s  key  member  of  the  team  to  help  us  remain  lsadsrs  in 
missiles  so  we  can  Influence  what  we're  buying  from  the  missile- 
makers. 


STATISTICIANS  IN  AIR  FORCE  ORGANIZATIONS 
Paul  R.  Rider 

In  the  first  place  let  me  tell  you  about  some  of  the  Air  Force  organi¬ 
sations  that  are  concerned  with  statistical  methods.  How  these  organi¬ 
sations  originated  and  developed  is  ably  described  by  my  colleague  and 
nfficemate  H.  Leon  Harter,  in  on  article,  "Statistics  in  the  Air  Frrcv 
Research  Progrant,  "  which  appeared  in  the  American  Statistician  for 
October,  1962  (Vol.  16,  no.  4,  pp.  23-24)  and  waa  reprinted  as  Appendix 
B  in  Harry  J.  Eisenman's  History  of  Mathematical  Statistics  Research 
at  the  Aeronautical  Research  Laboratories,  Office  of  Aerospace  Rassarch, 
United  States  Air  Force,  Washington  D,  C.  ,  1962.  Consequently  I  shall 
not  go  Into  historical  matters  here. 
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Two  element*  of  the  Office  of  Aerospace  Research  handle  practically 
all  of  that  organisation's  baeic  reeearch  program  in  etatletlce.  On*  of 
thee*  element*  ii  the  Air  Force  Office  of  Scientific  Research,  alec  lo¬ 
cated  in  Washington.  It  ha*  a*  it*  principal  function  the  awarding  and 
monitoring  of  contract*  and  grant*  for  baaic  reeearch.  The  aubject  matter 
oi  the  (tatiatic*  contracts  include*  nonparametric  methods,  order  statistics, 
estimation  theory,  distributions  of  statistical  functions,  design  and  analysis 
of  experiments,  decision  theory,  foundation*  of  probability,  limit  theorems, 
stochastic  processes,  Markov  processes,  and  combinatorial  analysis,  as 
well  a*  applications  of  probability  and  statistics, 

The  other  element  is  the  statistics  group  of  the  Applied  Mathematics 
Research  Laboratory,  a  component  of  the  Aerospace  Research  Laboratories, 
located  at  Wright- Patterson  Air  Force  Base,  Ohio,  This  element  performs 
most  of  the  in-house  basic  research  in  statistical  it  also  does  a  small 
amount  of  consulting  work  and  monitors  a  few  contracts  in  areas  closely 
related  tc  the  Internal  work.  These  areaa  include  design  and  analysis  of 
experiments,  estimation  of  parameters,  and  probability  theory.  Specific 
topics  being  studied  are  transformations  in  the  analysis  of  variance,  multi¬ 
ple  comparisons  tests,  multivariate  analysis,  the  us*  of  order  statistics 
and  functions  of  ord«r  statistics  (t.  g. ,  range  and  quaal-range)  to  obtain 
estimates  of  parameters,  mixed  and  truncated  distributions,  information 
theory,  circular  error  probabilities,  and  probabilistic  model*  for  applied 
mathematical  problems.  Member*  of  this  group  have  conducted  courses 
in  statistics  and  sponsored  mailers1  these!  by  students  in  the  Air  Force 
Institute  of  Technology,  although  they  are  not  at  present  engaged  in  any 
formal  Instructional  activities. 

The  Operations  Anu!y*U  Chiles  at  Headquarters  USAFa»*.e  statistical 
metnods  extensively  and  perform*  a  certain  amount  of  reeearch  in  applied 
statistic*. 

The  Air  Force  Systems  Command  makes  extsnslve  us*  of  statistics  a* 
a  research  tool  and  performs  a  fair  amount  of  research  in  applied  statis¬ 
tic*.  Among  the  various  divisions  of  AFSC,  probably  the  one  having  the 
most  nxteneive  program  of  statistical  research  is  the  recently  formed 
Bloa^troneutles  Division;  particular  mention  should  be  made  of  the  work  of 
the  School  of  Aviation  Medicine  located  at  Brook*  Air  Fore*  Base,  Texas, 
and  the  Aerospace  Medical  Research  Laboratories  et  Wright-Patterson  Air 
Force  Base,  Ohio.  At  the  School  of  Aviation  Medicine,  research  la  being 
performed  or  monitored  on  the  statistical  theory  of  epidemic*,  the  sampling 
distribution  of  the  characteristic  root*  of  Wishart  matrices,  the  analysis 
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of  repeated  measurements,  classification  statistics,  and  non-parametric 
oe verel- sample  taata.  At  the  Aerospace  Medical  Research  Laboratories 
research  has  baan  performed  on  distribution-free  teata  and  conaidarabla 
uae  haa  baan  made  of  the  design  and  analysis  of  experiments  In  their  tasting 
of  materials,  equipment,  rations,  physical  endurance,  etc.  In  the  Ballistic 
Miaaila  Division  of  AFSC  soma  applied  statistical  research  has  been  ad¬ 
ministered  by  the  Air  Force  Missile  Development  Center  at  Holloman  Air 
Force  Base,  New  Mexico,  and  by  the  Air  Force  Missile  Test  Center  at 
Patrick  Air  Force  Base,  Florida,  Naturally,  much  of  the  work  at  these 
two  centers  is  concerned  with  reliability. 

The  Air  Force  Logistics  Command  employs  statistical  quality  control 
and  acceptance  sampling  techniques. 

Contract  groups  are  the  Rand  Corporation  at  Santa  Monica,  California 
and  the  Institute  for  Air  Weapons  Research  at  the  University  of  Chicago. 

1  have  tried  to  sketch  t.  picture  of  the  present  state  of  statistics  in  the 
Air  Force  research  and  development  program,  and  now  I  shall  attempt  to 
say  how  l  think  the  situation  can  be  improved.  I  would  not  urge  any  change 
in  the  overall  organisation.  I  do  believe  that  many  more  statisticians  could 
be  used  in  all  of  our  fields  oi  endeavor.  We  could  use  more  research  statis¬ 
ticians.  Of  course,  it  is  impossible,  or  at  least  difficult,  to  force  the 
direction  of  basic  research,  but  some  effort  could  be  made  to  assemble  groups 
of  statisticians  with  common  Interests  in  problems  that  would  likely  have 
some  useful  application  to  Air  Force  problems  some  time  In  the  future.  1 
would  not,  however,  throw  out  any  project  simply  because  no  possible 
application  can  be  imagined  for  it,  even  in  the  far-distant  figure,  it  is  well 
known  that  •uius  theories  wh>>.h  at  one  time  considered  of  academic 

interest  only  have  subsequently  been  found  to  be  of  great  practical  utility. 

For  example,  group  theory,  once  regarded  as  of  no  conceivable  uCe  has 
beccme  extremely  important  in  the  design  of  experiments. 

But  mainly  I  should  like  to  see  more  statisticiane  employed  ae  consul¬ 
tant#  in  various  fislds.  These  persons  should  b*  familiar  with  the  parti¬ 
cular  fields  in  which  their  services  are  employed,  whether  thia  be  th« 
physical  sciences,  the  biological,  logistic,  or  other.  Ideally,  a  strong 
group  of  such  persons  with  practical  knowledge  and  sound  theoretical  training 
should  be  attached  to  every  Important  research  or  development  organisation. 
This  would  probably  not  be  feasible  becauee  of  the  difficulty  of  getting  a 
sufficient  number  of  qualified  statisticians.  The  next-best  plan  would 
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doubtless  i.j  to  have  *  central  bureau  oi  consultant*  whose  service*  would 
be  available  to  those  organisation*  desiring  them.  1  do  not  want  to  bring 
up  the  question  of  unifying  the  armed  services,  but  I  sea  no  reason  why 
the  advice  and  assistance  of  such  a  buroau  should  not  be  available  to  any 
organisation  within  the  Department  of  Defense. 

In  uummary,  1  think  we  need  more  statisticians  of  all  kinds,  and  al¬ 
though  I  am  connected  with  a  group  whose  primary  mission  le  basic  re¬ 
search,  l  believe  that  the  greatest  need  le  for  etatistlcal  consultants,  . 
those  who  can  give  practical  aid  on  important  research  and  development 
problems  needing  immediate  solution, 


DESCRIBABLE  CHARACTERISTICS  IN  A  STATISTICIAN 
William  Wolman 

"What  type  of  physicist  is  needed  in  a  laboratory"  or  "What  type  of 
astronomer  le  needed  for  an  obaervatory"  are  questions  which  are  rarely 
asked,  Why?  Because  the  prime  and  overriding  requirement  is  for  a 
scientist  with  ths  highest  technical  qualif  ication*.  In  considering  the 
question  as  to  what  type  of  statistician  is  needed,  we  are  dealing  with 
s  .'iT^what  of  a  different  problem,  What  makes  it  a  different  problem? 

To  enumerate  a  few  characteristics  regarding  a  statistician,  the  following 
points  come  to  mind: 

L.  He  must  deal  with  other  scientists  and  engineers, 

i.  Tho  field  i»  now  and  la  i  reiativeiy  unknown  scientific  discipline. 

Thu  re  are  many  pttople  in  the  profession  who  got  In  throu^!-.  the 
back  door  and  who  art  technically  not  up  to  par. 

4.  He  must  be  willing  to  communicate  and  listen  to  other  people's 
technical  problems. 

As  a  basic  requirement  the  statistician  must  have  adequate  technical 
qualifications.  I  would  Include  s  sound  foundation  in  the  design  and  analyais 
of  experiments  -•  th*  Analysis  of  Variance,  Multiple  Comparison  Methods 
and  knowledge  of  some  of  the  more  exotic  designs,  such  as,  "split  plot", 
"fractional  replication"  and  response  surface  technique*.  Th*  Level  of 
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understanding  »hould  be  that  which  is  described  in  such  excellent  booke  ee 
Kempthorne,  Brownlee,  Cochran  end  Cut,  and  Scheffi,  references  1,  2,  3, 
end  4,  A  knowledge  of  the  theory  of  estimation  including  confidence  inter- 
vela  end  the  determination  of  ennrees  of  variation  ae  measured  by  variance 
components  is  another  technical  area  of  importance  for  the  laboratory 
statistician.  Furthermore,  some  knowledge  of  the  various  types  of  fixed 
and  random  models  for  the  Analysis  of  Variance  is  important,  I  would 
like  to  recommend  that  if  a  laboratory  desires  a  statistician  that  they  hire 
a  professional.  Today,  there  are  at  least  a  dosen  first-class  universities 
which  have  programs  in  modern  statietical  theory  and  its  application,  It 
is  not  sufficient  to  transform  a  member  of  another  diacipllne  into  a  statis¬ 
tician,  Tha  common  practics  oi  using  a  "warmed  over"  engineer  ae  a 
statistician  can  only  lead  to  undesirable  consequences.  I  would  like  to  re- 
iLersta  that  t  am  not  epaaking  about  many  fine  statistician*  whose  original 
training  was  In  soma  othar  diaciplina  and  who  later  in  their  career  drifted 
into  statistics.  I  have  frequently  encountered  individual*  who  make  the 
statement,  "l  am  not  a  statistician"  and  then  enter  into  a  lengthy  dis¬ 
course  on  some  statistical  questions  and  consider  their  opening  disclaimer 
a  license  to  commit  every  conceivable  kind  of  technical  and  logical  error. 

In  closing  this  paragraph  I  would  like  to  point  out  that  tha  technical  requlra- 
ments  which  1  hava  enumerated  above  are  only  example*  and  should  be 
supplemented  with  a  sound  foundation  in  statistical  Inference  and  probability. 
Furthermore,  I  am  refarring  to  a  consultant  statistician  and  not  a  research 
statistician,  namely,  on*  who  doe*  research  in  the  theory  and  methods  of 
mathematical  statistics. 

Having  sound  technical  qualification*  is  not  sufficient  for  a  consults^4 
statistician  iu  tha  laboratory-  He  muat  be  able  to  communicate  and  speak 
the  language  of  his  callow  scientists  end  engineers,  bom*  understanding  of 
the  subject  matter  ia  also  mc..,datnry,  H#  must  certainly  find  out  what  the 
investigator  whom  he  ia  trying  to  help  wants  to  uncover  or  detain. In*,  For 
example,  if  it  ie  a  problem  of  determining  which  of  two  processus  are  super 
lor.  then  he  must  recommend  to  the  experimenter  a  procedure  which  will 
provide  at  least  a  50:30  chance  of  detecting  differences  of  a  magnitude 
which  are  considered  to  be  of  consequence,  A  knowledge  and  undar standing 
of  laboratory  techniques  and  metrology  is  certainly  a  most  daslra’  lt>  know¬ 
ledge  for  the  statistician  in  the  laboratory,  Wilton's  book,  reference  5, 
gives  an  account  oi  tome  of  the  function*  and  requirements  for  the  experi¬ 
menter  and  statistician  in  the  laboratory. 

As  a  consultant,  the  statistician  must  also  have  certain  personal 
characteristics  li  he  is  to  perform  hi*  Job  successfully.  He  must  hsve  a 
desire  to  help  his  fellow  ecientiste  end  he  must  be  able  to  communicate  with 
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them  freely.  His  attitude  should  not  b*  antagonistic  or  dominaering.  Ha 
should  realise  that  ha  la  performing  a  technical  support  function  which  ia 
uaually  not  in  the  limelight  of  an  operation.  I  have  found  that  if  the  experi¬ 
menters  In  the  laboratory  feel  that  you  have  something  to  sell,  namely,  that 
you  are  able  to  help  them  in  their  work  that  you  will  soon  be  in  a  position  of 
having  more  to  do  than  you  can  possibly  handle, 

Some  closing  remarks  as  to  the  organisational  location  for  a  statistician 
in  a  laboratory  follow:  The  statistician  should  have  easy  access  to  the 
experimenters  and  should  not  be  hindered  by  any  organisational  barriers, 

It  Is  usually  desirable  to  deal  with  the  experimenters  In  an  informal  manner. 
Constraints  in  terms  of  reporting  and  approval  procedures  should  be  held 
to  an  absolute  minimum,  For  instance,  keeping  track  as  to  ths  number  of 
contract!  a  statistician  has  with  his  collsaguss  is  virtually  meanlnglsss,  The 
contribution*  of  a  consultant  statistician  art  oftsn  very  difficult  to  measure 
and  may  camlet  of  an  acknowledgment  at  the  end  of  an  experimenter' •  paper 
in  a  technical  journal,  or  a  statement  of  appreciation  made  at  a  meeting.  I 
do  not  consider  the  formal  organisational  location  of  the  statistician  to  bn 
particularly  important  if  he  can  perform  hi*  functions  in  a  manner  ae  I  have 
outlined  above.  It  le  only  If  the  organisational  location  is  concerned  with  the 
remuneration  that  the  individual  is  to  receive  that  it  may  be  an  important 
question. 
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TYPES  OF  STATISTICIANS  NEEDED 
Donald  A.  Gardiner 

The  research  and  development  laboratory  need*  two  types  of  statis¬ 
tician  to  occupy  throe  kinds  of  position  on  the  statistical  consulting  Staff. 

I  would  call  the  two  types  of  statistician  tha  externa'  consultant  and  the  in¬ 
ternal  consultant.  The  external  consultant  is  the  at  .tis tic lan  who  talks 
with  the  scientist  or  subject  matter  specialist  about  his  problem  and 
attempts  to  work  out  a  solution.  The  Internal  consultant  is  the  statistician 
who  consults  with  statistic!  ans  and  mathematicians.  The  three  kinds  of 
position  may  beat  be  wplained  by  reference  to  the  accompanying  diagram. 


The  diagram  ahow  a  what  1  think  the  structure  of  a  statistical  consult¬ 
ing  staff  should  look  like.  The  circled  portion  represents  a  basic  "module" 
or  building  block  of  the  organisation.  I  have  shown  two  modules,  but  a  largo 
organisation  may  need  more.  A  smaller  laboratory  may  need  one  module 
or  only  a  part  of  one.  The  letter  aymbole  ar«  Interpreted  as  follows: 

MS  Math«~>alUtl  or  theoretical  statistician 

ES  Experimental  or  sopited  statistician 

S  Statistician,  cither  mathematical  or  experimental 

PR  Programmer 

Although  1  consider  the  programmer  a  basic  and  very  Important  part  of 
tha  organisation,  a  discussion  of  his  role  and  hie  characteristics  le  not  per¬ 
tinent  to  this  panel. 

The  external  consultant*  are  the  S  and  ES  statisticians.  Both  of  thee* 
people  treat  directly  will:  the  customer.  Therefore,  they  should  be  person¬ 
able,  they  should  meat  people  well,  and  have  the  ability  to  exprese  them- 
selves  clearly  in  a  non-mathematical  manner.  These  qualities  are  deelrabl* 
in  the  internal  conaultant,  too,  but  they  are  not  absolutely  necessary.  While 
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we  sr e  considering  the  internet  consultant,  it  might  be  well  to  point  out 
how  his  position  could  be  filled  in  the  case  of  the  laboratory  which  does 
not  have  a  full  statistical  staff.  I  think  that  in  this  case  the  services  of  an 
MS  statistician  might  be  obtained  on  a  part-time  basis- -perhaps  by  hiring 
a  consultant  from  a  university  for  a  few  days  each  month. 

The  S  and  £S  statisticians  should  have  a  acienes  background.  This  is 
probably  bast  obtained  by  taking  aBachelor  of  Science  degree  at  a  university. 
The  mathanatical  acientistician  would  need  mathematics  training  equivalent 
to  a  masters  degree  in  mathematics,  More  important,  however,  is  their 
education  in  statistics.  Both  the  S  and  MS  statistician  should  have  statis¬ 
tical  training  at  the  Ph,  □.  level  and  the  ES  statistician  at  the  masters 
level,  1  do  not  want  to  say  that  these  degrees  should  be  required,  However, 
the  levels  of  training  1  have  in  mind  can  be  obtained  today  only  in  the  graduate 
schools  by  courses  of  study  leading  to  the  Ph.  D.  and  masters  degrees. 

I  do  not  suscrlbs  to  the  idea  that  a  statistician  working  with  chemists 
should  have  intensive  training  as  a  chemist  nor  that  a  statistician  working 
with  engineers  should  have  training  as  a  engineer.  To  paraphasa  Qsorge 
Box  *  in  this  regard,  there  would  then  be  little  point  in  training  ohamists 
or  engineers  ••  only  statisticians.  But  I  would  require  that  the  statis¬ 
ticians  be  willing  to  ehow  their  ignorance  in  the  field  of  application.  To 
be  able  to  help  the  sr.isntlst  they  must  be  inquisitive  even  to  the  point  of 
asking  stupid  questions, 


*0,  E,  P,  Box,  "The  Exploration  and  Exploitation  of  Response  Surfaces) 
Some  General  Considerations  and  Examplss,  "  p,  16-60  in  Bjorn  strips, 
vol  10,  (1954). 


t 


AN  ANALYSIS  OF  FACTORIAL 
EXPERIMENTAL  DESIGNS 


L.  W.  Keeting 

U.  S.  Army  Micelle  Command 
Redetono  Arsenal,  Alabama 


ABSTRACT,  A  simple  technique  Involving  the  analytic  of  variant*  la 
propoaad  for  tha  Initial  evaluation  of  data  obtained  from  a  factorial  experi¬ 
mental  design.  The  application  la  presented  for  a  general  two. factor 
rectangle  that  could  result  from  any  factorial  deaign,  Equations  were 
developed  that  permit  tha  use  of  a  uniform,  simplified  work  sheet.  An 
experimental  design  using  the  technique  is  proposed  for  studying  the 
effaete  of  heat  on  the  insulation  barriers  separating  auxiliary  rockets 
from  other  mlsaile  components.  A  factorial  design  is  suggested  with 
four  factors  at  levels  4,  3,  3,  and  2,  "confounded"  in  the  multiple  inter¬ 
actions.  Tha  type,  thickness,  and  positions  of  insulation  materials 
and  tha  type  of  structural  material  supporting  the  insulation  are  con¬ 
sidered.  More  test  information  and  greater  precision  for  lees  teat  time 
and  equipment  are  tha  advantages  of  factorial  designs  of  the  type  suggested. 

A  numerical  example  illustrates  the  application  of  the  analysis  of  the 
variance  technique  at  marginal  condltiona  of  significance, 

SYMBOLS. 

A,  B,  C,  D  Positions  g 

Insulation  type  [ 

Insulation  thickness  - 

Levels  | 

Combinations 
Sum  of  squares 
Total 
Variants 

Data  point  or  measurement 

Support  material  type  of 
thickness 


F,  R 
o,  a,  b 
k,  m,  n,  p 
Q 
S 
T 
V 
X 

x,  y,  e 


i 
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SYMBOLS  (CONT,  ) 


d,  f, 

I,  II,  ■ 
(1),  (2), 


Degrees  of  freedom 
Factors 
Material  sets 

Estimated  standard  deviation 
of  a  population 


SUBSCRIPTS 

C 

E 

e 

i 

J 

R 

S 

T 

W 


Column 

Error 

Combination  measurement 

Column  measurement 

Row  measurement 

Rows 

Subtotal 

Total 

Within  groups 
Variable  1,  J,  or  e 


INTRODUCTION  The  use  ot  o  factorial  design  with  confounded 
multiple  interactions  can  improve  precision  and  reduce  the  cost  of  testing. 
The  various  combinations,  whsn  colisctsd  into  groups,  form  natural 
partitions  of  the  whole  program.  Teats  on  the  parts  can  often  ba  performed 
independently.  Thus,  different  batches  of  raw  materials,  pieces  of  lest 
equipment,  and  experimental  environments  can  be  assigned  to  the  parte. 

A*  factors  are  shown  to  be  insignificant,  ths  associated  results  and 
degrees  oi  freedom  can  be  combined  with  other  factors  to  mors  prs» 
ciiely  determine  their  effect,  The  investigator,  inconsequence,  can 
arhir  ve  a  systematic  examination  of  the  data  available.  • 
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Factorial  designs  or*  applicable  to  many  fields  of  *tudy  in  which 
obaarvations  arc  mad*.  Th*  principles  are  u*ed  extensively  in  biology, 
m«dicin*,  paychoiogy,  aconomica,  and  eoeiolsgy.  In  certain  field* 
chamiatry  and  engir.iering  th*  ua*  1*  morn  limited,  To  quota  Dr.  Youden 
(Reference  1):  "Among  chamlata.  th*  recaption  to  far  aceordad  thia 
experimental  device  ia  reminiscent  of  th*  dilamma  of  th*  young  man 
seeking  employment.  No  on*  will  hire  him  until  h*  haa  experience, 
and  he  cannot  get  thie  experience  until  aomaon*  given  him  a  job.  " 

The  conflict  between  th*  ute  of  formal  experimental  dealgne  aud 
the  "ahotgun"  approach  to  reaearch  problem*  la  wall  known.  Formal 
da  eigne  will  uaually  be  attempted  if  time  ia  available  to  aalect  or 
originate  them  and  properly  treat  the  reeulte.  For  th*  average  eden- 
tiat  or  engineer,  who  haa  only  a  nodding  acquaintance  with  atatiatical 
principle*,  the  digging  from  text  book*  for  a  method  to  analye*  a 
specialised  factorial  deaign  it  no  email  taak.  U  boil*  down  to  a  matter 
of  coat,  time,  and  ability,  but  not  necetearlly  naivenea*  on  the  part  of 
th*  inveatlgator. 

Th*  purpoa*  of  thia  paper  ia  to  chow  in  detail  a  eimpl*  application 
of  the  analyal*  of  variance  to  a  general  factorial  experimental  deaign, 

Thia  paper  ia  intended  for  th*  aci.ontiat  or  engineer  who  can  ***  ad- 
vantage*  in  confounding  but  haa  limited  time  and  reaourcee  for  an 
analyal*  of  thia  type,  Detailed  atep*  are  pre  tented  including  a  work 
aheet  and  praaentation  form.  Th*  method  of  application  1*  baaed  upon 
fundamental  expraaeiona  derived  in  eeveral  texta  (Reference*  2,  3,  4, 
and  5)  and  upon  th*  aolution  of  numerical  example*  in  Reference  1,  An 
example  t«  given  that  illustrates  the  method  for  a  specific  roaaareh  problem, 

METHOD.  In  some  euientifie  investigations  an  attempt  ir  made  to 
hold  all  fa'ctors  constant  except  th*  one  under  immediate  etudy.  Teat* 
ar«  run  on  th*  variable  undtr  immediate  study,  which  i*  then  held  con* 
arant  while  a  second  factor  ia  varied,  Th*  procedure  ia  continued  until 
all  pertinent  variable*  have  been  investigated,  Each  factor  (X,  IX,  1X1, 
and  XV)  may  be  varied  at  two  or  more  level*  (k,  m,  n,  and  p).  Th<* 
total  numbar  of  different  experimental  combinations,  Q,  ia 

Q  ■  k  m  n  p. 

Teat*  on  every  poaaibl*  combination  are  uaually  prohibitively  expensive 
and  time  consuming  or  poaalble  outside  the  scope  of  the  investigation, 
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An  investigation  that  Include*  the  study  of  the  effects  produced  by 
varying  two  or  more  factors  simultaneously  provids*  ths  setting  for  a 
factorial  experimental  design,  Those  designs  oiten  hav*  advantagss 
over  holding  constant  all  variables  except  one.  For  example i 

1.  The  number  of  test  runs  can  usually  be  reduced  without 
sacrificing  important  data. 

2.  The  precision  of  the  investigation  can  be  improved. 

}.  Ths  cost  and  complexity  of  test  equipment  can  be  reduced. 

4,  Investigations  ran  be  undertaken  for  which  it  la  impossible 
to  hold  constant  several  important  variables. 

The  advantages  accrue  from  confounding  factors  and  levels  and  eliminat¬ 
ing  certain  combinations  on  ths  basis  of  prior  knowledge  or  engineering 
Judgment. 

A  general  factorial  experimental  design  is  shown  in  Table  1,  Since 
the  da  sign  is  presented  primarily  to  illuatrata  the  application  of  a 
method,  no  combinations  havs  yet  been  eliminated.  Under  the  heading 
Trample,  to  be  found  a  few  pages  later  one,  the  design  is  expanded  and 
certain  combinations  are  eliminated. 

The  analysis  of  variance  is  usually  employed  initially  to  evaluate 
the  data  taken  in  a  factorial  experiment,  Ths  variance,  ae  used  hero,  le 
the  square  of  the  estimate  of  the  standard  deviation  of  the  population  (r  2) 
represented  hy  the  svj silmenuU  data.  Variances  are  calculated  of  groups 
of  data  associated  with  the  factoes  and  level*,  If  the  group*  are  all  from 
the  fume  population,  the  result*  will  be  essentially  the  eame.  Tlv*  sroups 
showing  s  saparnts  influanca  on  ths  data  will  have  higher  varlancos,  F 
table*  indic«  *  the  magnitude  of  the  difference*  at  various  confidence  or 
probability  levola.  The  theory  underlying  the  analysis  of  variants  and 
the  F  table*  ii  prevented  in  detail  in  References  1  and  2,  and  a  develop¬ 
ment  will  not  be  attempted  here.  However,  formulae  and  procedures 
are  given  to  simplify  ths  computations.  Fundamental  expression*  are 
given  in  Table  II. 


Table  II 
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BASIC  EQUATIONS  FOR  THE  ANALYSIS  OF  VARIANCE 


SOURCE 

SUMS  OF  SQUARES 

d.  f. 

COLUMN  MEANS 

Sc  =  T'mJ(xi — x---)1 

k-1 

ROW  MEANS 

*  km  X  ( x-j-  -  X--)2 

n-1 

INTERACTION 

SI“SS-SC-SR 

(k-1)  (n-1) 

SUBTOTAL 

®s  =  m  I  (xlj.-X...)* 

nk-1 

WITHIN  GROUPS 

SW“ST-SS 

nk  (m-3] 

TOTAL 

ST  =  Kxije-X...)1 

nkm-  3 

THE  EIFAESSION  FOR  THE  SUM  OF  SQUARES  OF  THE  COLUMN  MEANS  CAN 
IE  NEAIRAMEO  TO  SIMFLIFT  THE  CONFUTATIONS. 


sc*nml(x-t..  -  x  . . .)Z - nm  [  (  xL..-x...)2-*-  (x,..-  x...)2*... 


WHICH  It  SQUARED  TO  OBTAIN 


sc  »  nrr.  [x2..-Zx1..5i...  +  x,...]  + 
nm  ’L>it  -Zx£..R...+X?;.]  +. 
nm  [^..-Z^.X..^*..]. 


SUFDINI, 


Sc»nm>.  51:  -  Z  nm  X ...  £  X:..  +  knm  Xa... 


FROM  TABLE  I. 


nm  t- "  Z.  Ti..  ■  T..;.  ■  k»im  X... 

SQUAB  I N^  BOTH  SIDES  AMO Se ARRANGING, 


nm  I  X  f..  ■  f  Tp.  knm  j?1  ■  J  *.  . 

*■  t  L*  l  “Fim  Nnm 

•oi.TlTUTINO  EQUATIONS  <41  ANO  v'j)  IN  EQUATION  (S)IIVEB 


(I) 


(21 

(3) 

Ml 

(5) 


UNION  IS  MOAT  CONUNUtl  FOR  USE  IN  TNC  NORN  SHEET  FORM. 
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Sums  of  squares  for  rows,  subtotals,  and  totals  can  be  treated  in 
the  same  manner  to  obtain  the  expressions  in  Tables  III  and  IV. 

Table  III 

WORK  SHEET  FORM 


Table  IV 
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ANALYSIS  OF  VARIANCE  PRESENTATION  FORM 


SOURCE 

SUM  OF  SQUARES 

d.f . 

VARIANCE 

F  RATIO 

COLUMN  MEANS 

S  *  £  Ti  •  •  T...* 

C  Pi  run  knm 

k-l 

vc’-f?r 

y<z- 

VK 

. 

ROW  MEANS 

$R  t  Lm  - 

j-i  Km  knm 

n-i 

v«*fSr 

L  _ 

Vr 

VE 

INTERACTION 

s  -S  S  ~s 

X  S  c  R 

_Yx_ 

Vw 

SUBTOTAL 

Se  -f  t  Tn*.-T"‘ 

nk-1 

WITHIN  GROUPS 

nk(m-i) 

Vw\n£:i) 

TOTAL 

ST’i  t 

nk  m-l 

Table  IV  ia  the  conventional  form  for  presenting  the  results  of  an 
analysis  of  variance  The  F  ratios  are  the  final  computations  and  are 
compared  with  critical  values  found  in  F  tables  in  many  references  on 
statistics  (References  1  and  2).  If  an  F  ratio  exceeds  the  critical  value, 
a  significant  effect  from  the  factor  i«  proved  at  th«  confidence  leve* 
selected. 

The  analysis  of  variance  is  applicable  to  ar.v  dccign  for  experiments 
at  two  or  more  levels  provided  the*  following  assumptions  are  acceptable: 

1.  There  is  no  interaction  between  row  and  column  facto,  c 
at  the  levels  investigated. 

2.  The  data  relative  tr>  the  factors  have  a  homogeneous 
variance  and  are  from  populations  with  normal  distribu¬ 
tions. 

However,  moderate  violations  of  these  assumptions  change  the  analysis 
very  little  and  the  validity  of  the  assumptions  can  be  checked  from  the 
data  to  be  treated. 
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Th«  fir*t  ratio  to  t**t  ii  "Interaction"  over  the  "within  groupe" 
variance.  The  pretence  of  eignifioant  interaction  invalidate*  further 
tratmant  by  the  analyeie  of  variance,  and  other  melhodi  muet  be  ueed. 
Whan  the  critical  F  value  ie  exceeded  indicating  interaction,  the  following 
condition*  exi»t  *eparately  or  in  combination! 

1 .  The  row  or  column  factor*  are  producing  effect*  when 
teited  together  that  do  not  occur  when  the  factor*  receive 
the  aamo  teat  *eparately. 

2.  An  additional  factor  i*  of  *ufficlent  importance  to  be 
included  in  the  analyeie 

3.  The  item*  in  the  aubgroup*  are  not  drawn  at  random, 

if  no  interaction  i*  present,  the  »ur,  »  of  squares  for  the  "interaction" 
and  "within  group*"  *ource*  can  be  added  and  divided  by  their  total 
degree*  of  freedom  to  provide  a  more  inclusive  error  term,  V^,,  where 


v  a  *  w 
E  nkm  -  k  -  n  +  1 

The  if  ratio*  for  row*  and  column.,  Vc/Vr  are  now  more  .enaitive, 

A  is  suit  higher  than  the  critical  value  for  F  tablo*  indicate*  a  eignif leant 
influence  from  the  row  or  column  factor.  The  prcaence  cr  abeence  of 
inte*:.<  tion  i*  in  lteelf  an  important  result  of  many  inveetigatione. 


tv ^  test  for  homoffor.ilty  of  variance  require*  the  computation  of 
the  ratio  of  Sc  and  S_'to  A  critical  F  value  ie  computed,  and  the 

result?  are  compared  (Reference  2). 

An  example  Illustrating  thl*  method  of  applying  the  ennlyei*  ot 
variance  to  factorial  dealgne  follows.  One  caee  of  interaction  between 
two  factor*  1*  given. 


EXAMPLE.  A  factorial  design  1*  presented  with  four  factor*!  one 
at  four  lev*i*7~two  at  three  level*,  and  on*  at  two  level*.  The  analyeie  of 
variance  technique*  are  ueed  to  compute  the  deeired  probabilities.  Hypo¬ 
thetical  reiult*  are  u«ed  to  illustrate  the  analyeie  at  marginal  condition* 
of  significance. 
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Present  missilo  design*  usually  include  smell  auxiliary  rockets  that 
perform  various  functions  within  the  overall  mission.  Oas  generators 
exhausting  through  turbines  are  used  to  rotate  propellant  pumps  and 
electrical  generators.  In  some  designs,  the  generators  directly  press¬ 
urise  propellant  tanks  and  hydraulic  systems.  .Small  v  ket  motor* 
are  uaed  to  spin  certain  stages  of  the  missile  for  axial  *  ability, 

Vernier  motore  are  fired  to  adjust  speed  and  direction.  Ratiorockets 
used  on  planet  probes  decrease  the  speed  of  the  satellite  whon  entering 
the  desired  orbit.  The  auxiliary  rockets,  located  at  many  points  on 
the  mleaile,  vary  In  sine,  thrust,  and  burning  time  and  operate  from 
solid,  liquid,  and  gaseous  propellants. 

Firing  auxiliary  rockets  often  causes  severe  environmental  change* 
around  adjacent  misaila  components,  High  temperatures  result  from 
exhaust  gases  and  hot  motor  surfaces,  Corroeion  and  erosion  are  tome* 
times  caused  by  the  discharged  gaeee, 

This  example  is  a  proposal  for  calculating  the  probability  of  an 
influence  from  material*  of  adjacent  component*  in  controlling  the  heat 
from  auxiliary  rocketa.  The  Influence  could  originate  from  either  type, 
thickness,  or  position  of  insulating  materials  or  the  type  of  supporting 
structure. 

The  proposed  design  may  not  be  optimum  whon  a  free  choice  of 
tactors  and  levels  Is  permitted.  In  this  case,  the  number  and  arrange¬ 
ment  of  the  factors  and  levels  depend  upon  the  scope,  precision,  method 
of  ‘.citing,  and  future  test  programs.  Factors  and  levels  are  unique 
to.  a  particular  experiment.  In  general,  dreigns  with  all  factors  at 
the  same  level  are  stmplci ,  Other  poeeible  combinations  are  enumer- 
atod  In  Raferrncee  6  and  V. 

ASSUMPTIONS.  The  proposed  design  applies  to  a  particular 
Ir  vt stigation  chariots riied  by  the  following  aeeumptionst 

1,  Eight  eats  of  insulating  and  supporting  msterial  sample* 
are  the  maximum  that  can  be  placed  around  a  single  roc'-ot 
motor.  (The  arrangement  may  be  similar  to  that  shown 
in  Figures  1  and  2, ) 


Figure  2.  EXAMPLES  OF  MATERIAL  SETS 


2,  Thermocouple!  are  uniformly  installed  cm  the  surface  of 
the  supporting  material  on  the  aide  away  trom  the  motor 
(Figure  2). 


Figure  2.  THERMOCOUPLE  LOCATION  ON  A  MATERIAL  SET 


).  Thermocouples  are  located  at  the  same  four  poaltlono 
on  each  cat  of  material  samples. 

4.  Only  one  rocket  motor  will  be  fired  with  a  given  set  of 
material  samples.  Thermocouple  outputs  will  be  rerout¬ 
ed  until  the  samples  reach  thermal  stability. 

5.  Four  factors  at  levels  4,  3,  3,  and  2,  as  given  In  Table 
Vi  will  be  investigated. 

ARRANGEMENT 

For  an  inveatlgation  baled  upon  the  ebove  assumptions,  the  design 
in  Tebls  V  is  proposed  for  the  arrangement  of  factnro  and  levels, 
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Table  V 

PROPOSED  FACTORIAL,  DES1QN 


AMJUSikl  MitttM 


FACTOR 

. 8YMML  . 

POSITION  -  INSULATION  AND  SUPPORT  MAXIMAL 

A,  1,  C,  0 

H 

TVPC  -  INSULATION  MAXIMAL 

r,  r 

1 

THICKNESS  -  INSULATION  MATERIAL 

o,  •,  b 

3 

THICKNESS  OR  TYPI  •  SUPPORT  MATERIAL 

»,  y,  i 

> 

n 

y.  .  . 

I 

0  *  -  b 

>  a  b 

o  i  b 

A 

*0)  *(») 

*<l> 

*(4) 

P  ■ 

*0)  *(«) 

<<*> 

*<«> 

0 

*(1)  *(t) 

*(*> 

*<«> 

0 

*(l>  *<1) 

*(S) 

.  *(•) 

A 

l(»)  *0> 

*(S) 

■ 

*(l> 

l(i> 

*(•> 

n  c 

*<#> 

><l) 

Xf»> 

0 

*(»> 

(<l>  *<T! 

*(S) 

The  number!  In  parentheete  refer  to  the  eight  eeta  of  insulation 
and  support  material  sample*  grouped  around  the  rocket  motor  (Figures 
1  and  2),  Four  thermocouples  are  uniformly  located  on  each  set  repre- 
renting  the  four  positions,  A,  B,  C,  and  D  (Figure  1). 

The  analysis  of  variance  for  the  factorial  design  shown  in  Table  V 
is  summarised  in  Table  VI  . 


Table  VI 


ANALYSIS  OF  VARIANCE 


NUMERICAL  COMPUTATIONS 

The  following  numerical  example  (Tables  VII  through  XIII)  la  given 
to  illustrate  the  initial  method  of  antlyaia  for  a  factorial  experiment. 
The  eompoaite  deaign  ia  broken  down  into  simple  two-factor  rectangles 
suitable  for  the  analyais  of  variance  technique.  The  hypothetical  data 
are  thermocouple  readings  in  degrees  F, 
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Table  VIII 

ANALYSIS  OF  VARIANCE  FOR  ABCD  VS  xyz 


24 

READINGS 

12  TOTALS 
durl?£ates 

RON 

TOTALS 

COLUMN 

TOTALS 

GRAND 

TOTAL 

SQUARE 

AND  SUM 

2,992.894 

6.982,360 

23,889,370 

17,916,460 

71,605,444 

0IVI0E  BT 

1 

2 

8 

6 

24 

QUOTIENT 

2.992. BOM 

2,991,180 

2,986, 171 

2,986,076 

2,983,560 

SUBTRACT 

2,983,660 

2,983,560 

2,983,560 

2.983,560 

SUM  OF 
SQUARES 

9,334 

7.820 

2,5!  1 

2.516 

. . — _ _ 

SOURCE 

SUM 

OF  SOUARES 

4.1. 

VARIANCE 

F 

COLUMNS 

25  IB 

3 

338 

3.59 

ABCD 

ROWS  xyz 

261 1 

2 

1305 

5.60 

INTERACTION 

2493 

6 

415 

2.9? 

SUBTOTAL 

1 1 

WITHIN 

1714 

12 

142 

GROUfS 

TOTAL 

9334 

23 

V,  *  2493  »  1714  =  4207  •  233 
1  6  •  IZ  18 


THE  EFFECTS  OF  ABCO  ANu  xyz  AH£  5t6NiriCANT  AT  THE  CONFIDENCE  LEVEL  Of  0  A*’ 
BUT  NOT  AT  OM  SINCE  3.!6<  5  59<  5.09  AND  3.33  <  5.Q0«  S.OI. 
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Tabic  IX 


EXAMPLE  OF  INTERACTION  BETWEEN  ABCD  AND  xyz 


- 1 - - 

_  B  . 

C 

0 

361 

372 

320 

357 

X 

329 

360 

315 

327 

690 

732 

635 

684 

2741 

361 

359 

375 

310 

y 

375 

358 

361 

329 

738 

717 

736 

639 

2828 

373 

355 

359 

370 

l 

379 

351 

359 

347 

753 

706 

718 

717 

2893 

2IIB 

2155 

2089 

.  .zm _ ! 

24 

” 

12 

■“*1 

GRAND 

READINGS 

TOTALS 

ROWS 

COLUMN 

TOTAL 

SQUARE 

2,992,894 

5,982,360 

21,380,114 

l>, 913,230 

71,805,444 

AND  SUM 

DIVIDE 

1 

C 

A 

L» 

A 

24 

QUOTIENT 

2,992,894 

2,991,180 

2,985,014 

2,985,538 

2,983,560 

SUBTRACT 

2,983.560 

2.982.W 

2,983,560 

2,983,560 

SUM  AND 

0 , 

7,020 

1,454 

1.97R 

SQUARE 

_ 

.J _ 

1- . .  .i  .  nrmh.. 

SOURCE 

me  •  ■innW’.i 

SUM  OF  SQUARES 

d.l. 

VASIAi.CC 

r~v 

COLUMNS 

1978 

3 

659 

ROWS 

1454 

2 

727 

INTERACTION 

4138 

6 

690 

’  s 

SUBTOTAL 

7620 

II 

1 

WITHIN  GROUPS 

1714 

12 

142 

TOTAL 

9334 

c3 

1 

_ 1 _ 

INTERACTION  IS  SIGNIFICANT  ABOVE  THE  99-PERCENT  CONFIDENCE  LEVEL  SINCE 
4.91  >  4.82  FOR  6  AND  12  DEGREES  OF  FREEDOM. 
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Table  XI 


ANALYSIS  OF  VARIANCE  FOR  ABCD  VS  oab 


iU 

READING* 

12  TOTALS 

OF 

DUPLICATES 

RON 

TOTALS 

COLUMN 

TOTALS 

GRAND 

TOTAL 

SOU ARE 

AND  SUM 

e, BIN, 227 

5,520,071 

22,NNI ,1*85 

13,843,631 

67,289,200 

DIVIDE  BY 

1 

2 

8 

6 

24 

QUOTIENT 

2,8)4,227 

2,810,035 

2,806,2NS 

2.807,271 

2,803,718 

SUOTRACT 

2,003,718 

2,803,718 

2,803,718 

2,803.718 

SUM  OF 
SQUARES 

10,500 

6,317 

1,530 

3,553 

SOURCE 

SIM  OF  SQUARES 

d.f. 

VARIANCE 

F 

COLUMNS 

3553 

3 

TO 

3.93 

ABCD 

ROUS  oab 

miA _ 

2 

765 

2.54 

INTERACTION 

I23N 

6 

205 

0.50 

SUITOTAL 

6317 

II 

WITHIN 

BROUPS 

UIB2 

12 

340 

TOTAL 

10600 

23 

Vg « im,}juaz. .  3oi 


THE  FACTO*  A SCO  IS  SIBNIFICANT  AT  THE  05-  TO  00-PERCENT  CONFIDENCE  RANGE. 
ROMS  AND  INTERACTION  ARE  NOT  SIGNIFICANT. 


Table  XII 

PROPOSED  FACTORIAL  EXPERIMENT 


4  x  3  x  3  x  2  NUMERICAL  EXAMPLE 


Table  XIII 


ANALYSIS  OF  VARIANCE  FOR  ABCD  VS  FR 


16 

READINGS 

8 

DUPLICATES 

ROW 

TOTALS 

COLUMN 

TOTALS 

GRAND 

TOTAL 

SQUARE 

AND  SUM 

1 ,027.060 

3,8*15,030 

15,363,820 

7,601 ,073 

30,724,840 

DIVIDE  BY 

1 

2 

8 

4 

16 

QUOTIENT 

l  ,927,069 

1 ,022,060 

1,020,1178 

1,022,768 

1 ,920,303 

SUBTRACT 

1.020,303 

1.020,303 

1,020,303 

1,020,303 

SUM  OF 
.SQUARES 

6,766 

2,666 

175 

2,465 

SOURCE 

SUM  OF  SQ 

d.f. 

VARIANCE 

F 

COLUMNS 

2465 

3 

822 

2.10 

ROWS 

175 

1 

175 

0.47 

INTERACTION 

25 

3 

8 

.02 

SUBTOTAL 

2666 

7 

WITHIN 

GROUPS 

4100 

a 

512 

TOTAL 

6766 

15 

VE  *  375 

THE  CRITICAL  F  VALUE  FOR  3  AND  II  d.f.  IS  3. SO  AT  05  PERCENT. 
SC  SIGNIFICANCE  OF  ANY  FACTOR  IS  SHOW  BY  THE  F  RATIO. 


In  the  proposed  design,  two  to  four  data  points  are  available  at 
each  combination  of  the  factors.  The  points  can  be  used  as  replications 
to  check  for  interaction.  However,  the  points  are  uoi  true  replications 
until  the  secondary  factors  in  the  combination  are  proved  insignificant 
l for  instance,  F,  R,  o,  a,  and  b  in  the  analysis  of  the  primary  factors 
ABCD  vs  xvz).  When  secondary  factors  are  significant,  the  test  for 
interaction  it,  valid  but  less  sensitive. 

As  factors  are  shown  to  be  insignificant,  the  associated  results  ana 
degrees  of  freedom  can  be  combined  with  other  factors  to  more  precisely 
determine  their  effect.  For  example,  in  the  proposed  design,  if  the 
insulation  thickness  factor  at  levels  o,  a,  and  b  is  shown  to  be  insignifi¬ 
cant,  three  data  points  for  each  combination  instead  of  two  become 
available  for  analyzing  the  remaining  factors.  Insignificance  of  jr.,  y 
and  z  must  also  exist  to  provide  extra  data  points  fur  ABCD  vs  FR. 

The  probability  of  an  influence  from  material?  of  adjacent  compo¬ 
nents  in  controlling  heat  from  auxiliary  rocket  motors  can  be  determined 
from  the  F  tables.  Vaiues  selected  l'rorr,  the  tables  that  bound  the  F 
ratio  are  computed  for  the  material  factor.  The  probability  is  then 
obtained  by  interpolation  of  the  corresponding  headings  of  the  F  table. 
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CONCLUSION,  Since  the  use  of  formal  experimental  designs  and  other 
statistical  principles  is  limited  largely  by  funds,  time,  and  capabilities  of 
the  investigator,  simplified  methods  of  analysis  should  be  developed  for 
those  not  thoroughly  acquainted  with  statistic  r,  One  approach  hae  been 
attempted  by  orienting  a  computational  procedure  with  fundamental 
expressions  for  the  analysis  of  variance  and  applying  the  tecnnique  to 
factorial  designs.  An  example  is  presented  illustrating  the  technique 
for  a  research  problem. 

The  use  of  auxiliary  rocket  motor*  on  missiles  often  create*  space 
and  weight  prooiems  In  insulating  adjacent  component*  from  hot  motor 
surfaces  and  exhaust  gases.  An  experimental  d«“!jr  1s  propoeed  for 
computing  the  probability  of  ,m  effect  from  material  factors,  such  ae 
insulation  type,  thickness,  position,  and  support  material  type. 

The  arrangement  of  factors  and  levels  Is  based  upon  limited  test 
conditions,  including  eight  sets  of  materials,  four  thermocouple  positions, 
two  types  and  three  thicknesses  of  insulation,  and  three  types  of  support 
material.  The  arrangement  provides  the  following  characteristics; 

1.  The  four  factors  can  be  varied  simultaneously. 

2.  Two  to  four  data  points  at  each  combination  of  primary 
factors  are  available  as  replications  to  test  for  interaction, 

3.  Ae  frctors  are  proved  insignificant,  the  associated  results 
can  be  combined  with  other  factors  to  increase  the  sensi* 
tivlty  of  the  remaining  analysis, 

4.  Tl.a  inherent  advantages  of  general  factorial  designs  apply, 
such  as  a  natural  purtttion  of  the  whole  test  progr»m, 
reduction  in  the  number  of  test  runs,  and  an  improvement 
in  scope  and  precision. 
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RELIABILITY  ESTIMATION  FOR  MULTI- COMPONENT  SYSTEMS 


J.  R.  Knin 

,  Ballistic  Research  Laboratories 
Aberdeen  Proving  Ground.  Md. 


INTRODUCE  ion,  xh«  U.  S.  Army  muit  maintain  a  world-wide  stock- 
pile  of  war  reserve  ammunition  for  both  combat  and  training.  Thle 
ammunition  must  be  kept  in  a  conetant  state  of  readineeei  and  it  le. 
therefore ,  Important  that  the  quality  and  reliability  of  the  etoekplle  be 
aatlnygted  periodically. 

^,'The  Surveillance  Oroup  of  the  Belli etic  Reeearch  Laboratories  periodi¬ 
cally  analyses  the  results  of  taste  conducted  on  samplee  taken  from  the 
/  stockpile.  In  the  case  of  conventional  ammunition  the  items  are  usually 
tested  ballletically  by  means  of  some  type  of  statistical  design,  and  the 
samples  are  destroyed  or  consumed  in  the  teste.  In  the  case  of  nuclear 
ammunition,  It  is  usually  impractical  to  evaluate  the  quality  and  relia¬ 
bility  of  the  stockpile  through  ballistic  tests.  Therefore,  laboratory 
test  data  are  obtained,  where  the  major  components  of  the  particular 
item  are  tested  statically  and  usually  non -destructively.  In  these  teste 
each  component  is  tested  as  an  individual  item,  and  it  is  assumed  that 
the  functioning  of  one  component  is  Independent  of  the  functioning  of  the 
remaining  components,  once  it  has  been  removed  from  the  system  lor 
testing  (i.  e. .  although  it  le  true  that  the  functioning  of  one  component 
will  generally  depend  upon  the  functioning  of  other  components,  the  in¬ 
herent  capability  of  its  functioning  as  a  component  within  itself  still 
exists). 

The  results  of  the  Individual  component  teste  must  then  be  analysed 
,.nd  combined  in  order  to  estimate  the  reliability  the  system  and 
also  to  place  an  interval  about  thle  estimate  which  will  yield  some 
specified  degree  of  confidence  that  the  true  reliability  Ueo  within  this 
Interval.  A  number  of  eolutlons  for  reliability  estimation  have  been 
developed  and  appear  In  the  literature.  However,  none  of  theee  solution* 
le  satisfactory  tor  the  type  of  problem  with  which  thle  organisation  le 
confronted,  It  wae,  therefore,  neceseavy  to  deviea  a  system  for  esti¬ 
mating  reliability  that  ie  highly  flexible,  feet  and  simple)  and  it  is  felt 
that  the  method  outlined  in  thle  paper  satisfice  these  conditions,  Although 
this  system  depends  upon  access  to  a  high  spaed  digital  computer,  it  dps* 
have  the  advantage  of  flexibility,  speed  end  simplicity  once  the  original 
problem  has  been  programmed. 
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DISCUSSION.  Determination  of  the  reliability  of  eaC-h  type  of  component^ 
is  relatively  straightforward.  It  is  first  required  that  a  decision,  based 
on  the  test  results,  be  made  on  each  item  tested  as  to  vvhether  it  is  a 
success  or  a  failure.  Once  a  decision  has  been  reached  on  each  item, 
the  number  of  successful  components  of  a  particular  typ?  (&)  follows  a 
binomial  distribution.  The  point  estimate  of  re  liability. for  each 
type  of  component  is  p  -  ^  where  n  i 3  the  number  tested.  A  con¬ 
fidence  interval  may  then  be  placed  on  p  using  statistical  procedures. 

However,  for  stockpile  reliability  testing  it  is  of  primary  interest  to  de¬ 
termine  the  reliability  of  the  entire  system.  This  relationship  caX*>® 
expressed  in  the  form  of  a  block  diagram  where  each  block  repreaertt# 
a  specific  component  or  part  tested.  (Two  or  more  blocks  could  repre¬ 
sent  the  same  type  of  component  where  redundancy  is  built  into  the 
system.  ) 


For  instance,  a  system  of  components  might  be  arranged  in  the 
following  manner 


where  B^  and  are  the  same  types  of  components.  If  the  reliabilities 
of  these  types  of  components  were  established  at  Pj,  p-,  and  p,  respective 
ly,  .hen  the  reliability  of  (R)  of  the  system  would  simply  be 


R  - 


Pi 


l  r.D, 


2 

-  P? 


1  -  £  type  of  component  could  be  a  timer,  radar  fuze,  etc. 
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Nuclear  warheads,  of  course,  are  much  more  complicated  than  the 
above  »xamplei  and  it  la  not  alwaye  possible  to  write  the  reliability 
equation  throug'  ‘v.spection  of  the  block  diagram.  The  following  la  a 
sample  of  a  bloc*;  diagram  whose  reliability  equation  cannot  be  readily 
obtained. 


In  order  to  compute  the  reliability  of  thle  syatem  of  components  It  is 
necessary  to  enumerate  all  combinations  of  suocees-fhllure  possibilities 
for  components  B  through  E.  Letting  a  success  be  represented  by  a  1 
And  a  failure  by  a  0  the  reliability  of  the  syatem  may  be  computed  in  the 
following  manner: 


60S 

Componente 

B  Cj  Cz  Dj  D2  Ij  Ej 
111  11  11 


1111110  PB  PC  PD  ***  8 

111110  1  PB  PC  P?>U  ‘  pK)  p*  8 


111  1100  PB  pd  (1  ‘  p*>*  F 

111  10  10  PB  PD<1  *  PD>  PI<1  *  8 


l  n  0  0  0  0  0  PB  (1-PC)2  (1  •  pD>2  0  -  p»)*  p 

0  0  0  u  0  0  0  (1  -  Pc)  (1  -  Pc)2  (1  -  P^,2{1  -  V 

All  probability*  that  yield  eucccieeee  ihoula  be  added,  end  the  reliability 
of  the  eyetem  can  then  be  written 

R  "  PAPE  lPD  12  ‘  PDPE>  +  PBPC{2  ‘  PG> 

+  PBPCPD  [2Pe'PD  *  +  PC<2  ‘  pD>  -  2  }  • 


Deiign  of  Experiments 


609 


The  reliability  of  the  lyitem  can  now  be  computed  from  the  above  equation. 
Thi»  ►.'temple  ie  one  nf  the  laait  complex  syetema  that  hae  been  encountered 
in  practice.  Moat  ayateme  require  thouaanda  of  enumaratlone  reaulting 
in  reliability  equationa  covering  eeveral  pagea, 

The  problem  of  placing  a  confidence  interval  on  R  la  diffloult.  The 
number  of  aucceaaea  (ft)  aaeoclated  with  each  value  of  $  hat  a  binomial 
distribution  and  a  confidence  Interval  may  be  placed  on  each  true  p  uaing 
tHl a  diatributlon.  In  order  to  place  a  confidence  interval  on  R,  however, 
it  ia  alao  naceaaary  to  utiliee  the  diatributlon  of  the  individual  ft1*,  aince 
R  ia  a  function  cf  x't, 

If  it  can  ba  aaaumed  that  $  it  a  random  variable  and  that  the  diatri* 
button  of  9  !■  known,  then  random  eamplae  can  be  drawn  from  theee 
diatributlon  a  and  aubatttutad  into  the  reliability  equation  and  random 
value*  of.  R  can  be  obtained.  The  diatributlon  of  R  can  bo  eatlmated, 
and  a  confidence  interval  can  be  conatructed  which  will  produce  aome 
maature  of  aaeurance  that  the  true  reliability,  R,  Ilea  within  the 
interval. 

The  diatributlon  of  $  (given  p)  may  be  repreaented  aa 
g<  Pi  P)  pn^(l-p)n(l'P> 


where  p  ■  0,  1/n,  2/n,  ,  .  .  ,  1,  However,  it  ia  not  poeaible  or  deairaUe 
u»  bample  the  abuvu  di  kirll  'tion  for  value*  of  $  aince  ■  x/n  muet  be  tub- 
atttuted  lor  p  and,  for  tlu.  ucae  where  ft  »  0  or  ft  «  n,  g($  i  p)  -  0. 

Thl*  ia  a  reault  of  the  aaamuption  that  p  •  ft/n,  and,  of  oourae,  if  p 
equal*  either  0  or  1  there  can  be  no  aampllng  error,  in  fact,  .'.r  nnv 
v«lue  of  ft  th*  variation  introduced  would  be  entirely  due  to  *empllng  , 
and  any  confidence  intervale  generated  would  not  be  realistic  alnco 
the  error  involved  in  eatlmating  p  would  be  Ignored. 

Confidence  interval*  may  ha  placed  on  an  individual  p  at  the  i«v*l  of 
confidence  by  ueing  the  binomiel  distribution  in  tha  following  manner. 
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<0 


VC 

1 


u  ■  £1)n'x  -  (i  ■  «o n 


(upper  toil) 


d  ■•()  /  l 


(lower  toll) 


x  -  0,  l . n. 

By  eolving  the  above  equation  at  specified  levels  of  •(,  it  Is  possible 
to  construct  an  interval  such  that  100 af  %  of  all  Intervals  constructed  in 
such  a  manner  will  contain  p.  It  is  also  possible  to  generate  values 
of  $  over  a  continuous  range  from  the  shove  equation  by  varying  •<  . 
However,  it  is  not  possible  to  construct  intervals  in  the  vicinity  of  ft/n 
where  o(  la  small  or  where  (1  -•»()/ 2  is  near  or  equal  to  0,  5,  In  fact, 
for  (1  -  flf  )/ 2  ■  0.  8,  where  one  would  expect  the  Interval  to  be  a  point, 
two  values  of  $  are  obtained  depending  upon  the  equation  used.  This 
result  becomes  intuitively  apparent  when  it  is  considered  that  equation 
(1)  gives  a  value  of  p.  for  which  (1  »ei)/2  (100)%  of  the  distribution 
will  he  ft  or  lees  ana  equation  (Z)  gives  a  value  of  £  for  which  ( 1  -•(  )/2 
(100)%  of  the  distribution  will  be  S  or  more.  Therefore,  where 
(1  -A  )/ 2  ■*  0,  S,  80%  of  the  distribution  must  be  ft/n  or  more  in  one 
case  and  80%  must  byft/n  or  less  in  the  other.  This,  of  course,  is  a 
contradiction  caused  by  the  fact  that  the  binomial  distribution  is  deaerate 
with  ."aspect  to  x,  but  $  take*  on  continuous  values, 

Since  equations  (1)  and  (2)  are  accurate  at  the  tails,  it  was  decided  to 
construct  a  distribution  function  using  these  tails  and  eomplsting  tl.u 
center  with  a  rectangular  distribution.  This  distribution  function  is 
In  eifvet  a  function  of  a  "created"  random  variable  and  has  the  pmpwty 
that  the  probability  of  selecting  a  sample  between  certain  limits  from 
this  distribution  is  equal  to  the  probability  that  p  U«e  within  that 
interval,  or 
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We  shall  cell  thle  function  f  (ft),  end  the  vert  Able,  of  course,  le  $ 
which  iu  now  e  continuous  variable  ranging  from  0  to  1.  ft  is  now 
possible  to  generate  values  of  $  and  substitute  them  into  the  reliability 
equation. 

Once  specific  values  of  and  are  substituted  into  aquation  (9), 

the  relationship,  of  course,  becomes  nonsense  since  obviously  p  either 
Her  within  specific  limits,  or  it  does  not.  However,  although  (6)  is  not 
in  this  sense  a  true  probability,  it  is  a  measure  of  confidence  in  the 
truth  of  the  statement  on  the  left  of  (5)|  and,  therefore,  the  relation¬ 
ship  1s  considered  to  be  sufficient  Justification  for  the  use  of  the  distri¬ 
bution  described  above. 

Actually  equation  (2)  may  be  used  to  select  samples  of  $  ranging  from 
0  to  (ft  «  l)/n  and  equation  (I)  may  be  used  for  aamplea  of  £  ranging 
from  (Si  +  l)/n  to  1.  The  interval  from  (ft  •  l)/n  to  (ft  +  l)/n  may  ba 
covered  by  a  rectangular  distribution  whoee  height  must  be  determined 
so  diet  tits  area  under  the  entire  distribution  will  be  1.  Figure  1  is  an 
example  of  such  a  distribution  funotion  for  e  particular  value  of  ft  and  n. 

This  distribution  is  actually  the  combination  of  two  beta  distributions 
and  a  rectangular  distribution.  It  is  s  result  of  tha  fact  that  the  beta 
distribution  integrates  to  the  binomial. 

t*  ( l  -  if  dt  ■  l*(yi  ,  4  ) 


oj  +  +  1  »  1 

y)  +  1. 


(?) 
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Similarly  «.nce 

4 - 


then 

(4) 

n 


(£>  P*(l  *  P)n'x-  f(pi  A-l.  n-ft). 


Therefore,  equation  (4)  may  be  used  for  the  lower  tall  of  the  distribution 
using  only  that  portion  of  the  curve  from  0  to  (ft  •  l)/n,  and  aquation  (3) 
may  be  used  for  tha  upper  tall  uaina  only  that  portion  of  the  curve  from 
(ft  +  l)/n  to  1.  The  Interval  from  (x  •  l)/n  to  (ft  ♦  1  )/n  will  be  covered 
by  a  rectangular  dletributlon  whoee  height  will  be  determined  ao  that 
tha  total  area  under  the  distribution  will  be  equal  to  one.  (If  only  the 
lower  limit  on  R  ii  desired,  equation  (4)  may  be  used  for  the  entire 
dletributlon)  similarly,  if  only  the  upper  limit  ie  deelred,  equation  (3) 
may  be  uead. )  Therefore,  a  dletributlon  can  be  eat  up  for  each  compon* 
ent  or  part  depending  upon  the  number  tested  and  the  obosrved  number  ot 
successes. 

In  the  even*1  'hit  ft  *  u  or  tnet  ft  ■  n  for  a  large  percentage  of  the 
components,  It  <■  poeeible  that  the  point  eetimato  of  reliability  (ft  will 
fall  outside  tha  confidence  interval.  This  results  from  tha  blnomUl 
dletributlon  where,  for  instance,  if  »  l  any  two-sided  confidence 
interval  (except  where  •<  ■  1)  will  not  include  Should  thle  eitu- 
etion  occur,  only  a  one-sided  interval  should  be  given,  (ft  It  quite 
possible  that  a  one-eided  interval  may  in  any  event  be  of  more  Interest 
than  a  two -elded  Interval. ) 

Through  the  uso  of  high  epeed  digital  computer*  each  of  thee*  dis¬ 
tributions  oan  be  randomly  sampled  and  thousands  of  values  of  ^ 
can  be  generated  for  the  l^1  component,  If,  for  instance,  1,000 
valuta  of  ere  randomly  selected  for  each  in  the  reliability 

aquation,  and  if  these  values  are  randomly  placed  in  thnt  equation,  then 
1,000  values  of  a  can  be  generated. 
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A  function  of  ft  con  bo  approximated  by  plotting  those  1,000  values  on 
a  histogram.  Confidence  limits  can  then  be  placed  on  A  olthar  by  . 
fitting  a  curve  to  the  histogram  or  by  using  the  generated  values  of  R 
directly.  The  sampling  error  involved  can  be  decreased  by  increasing 
the  number  of  values  generated.  It  is  no  problem  for  the  BRLKSC 
(Ballistic  Research  Laboratories  Electronic  Scientific  Computer)  to 
generate  even  10,000  values  of  for  each  distribution. 

A  number  of  trial  runs  for  ths  previous  example  have  bean  generated 
by  the  computer  and  figure  2  shows  sampls  histograms  that  hava  resulted 
for  both  individual  ft't  and  for  R  using  ft  ■  90  and  n  ■  100.  Figure  Ilia 
and  111b  are  histograms  generated  where  ft  »  12  and  n  *  20  and  where  R 
represents  the  reliability  of  a  simple  system  of  three  oomponente  in 
parallel.  It  le  anticipated  that  the  system  can  be  Improved  in  the  future 
by  programming  the  machine  to  fit  curves  to  the  hiitogram  and  to  integrate 
undor  these  curve*  in  order  to  produce  confidence  intervale. 

CONCLUSION. 

1.  The  objective  of  flexibility  has  been  satieties  in  that  a  point 
estimate  of  reliability  and  a  confidence  interval  about  that  ostimato  can 
bo  obtained  for  almost  any  eyatem  regardless  of  the  number  of  tests 
performed  on  the  individual  components  or  of  the  success  ratios  obtained. 

2.  The  objective  of  speed  end  simplicity  have  been  satisfied  since  it 
ie  possible  (or  the  mechlne  to  construct  an  Interval  in  e  few  minutes 
The  only  manual  operation  required  Is  to  punch  the  estimates  of  p  on 

<  -rds  end  place  these  cards  In  the  mechlne. 

3.  It  it  understood  ther  curtain  liberties  hevs  been  taken  in  creating 
a  random  variable  eueh  as  p.  However,  it  is  fell  that  thie  assumption 
1*  Justified  by  the  results  and  by  aquation  (3), 

4.  Obviously  the  distribution  used  for  these  estimates  is  not  the  best, 
particularly  in  the  vicinity  of  ft/n,  A*  is  generally  the  case,  thie  difficulty 
is  most  pronounced  for  smaller  sample  siees.  However,  the  imports*.1 
portions  of  these  distributions  ere  instrumental  in  determining  tus  telle  of 
ths  distribution  of  R,  Therefore,  in  only  rare  cases  would  th*  rectangular 
portion  of  the  distribution  be  important,  fBpaelel  instructions  to  the  machine 
are  anticipated  where  ft  -  1  ■  n  or  where  ft  *  n,  These  instructions 

will  slimlnete  the  rectengulsr  distribution  for  these  easel.) 
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STATISTICAL  ST  J  O  Y  OF  ACINC  CHARACTERISTICS 


OF  ARTILLERY  MISSILES 


Raymond  Bril 

Surveillance  Group,  Ballistic  Research  Laboratories, 
Aberdeen  Proving  Ground 


INTRODUCTION ,  Among  the  various  methods  employed  by  the 
Surveillance  Group  in  investigating  the  .'j.ini  characteristics  of  Artillery 
Missiles  are  rocket  motor  static  tests,  chemical  end  physical  tset  of 
propellant,  fully  instrumented  proving  ground  flight  teats  and  evaluation 
of  service  practi  •«.  (I rings , 

A»  indicated,  evaluation  of  service  pricticf  firing*  to  one  of  the  methods 
employed  by  the  Surveillance  Group  in  investigating  aging  characteristics, 
These  firings  are  conducted  by  troop  units  for  training  purposes,  Although 
these  firings  are  not  fully  Instrumented  flights,  sufficient  data  is  obtained 
for  performance  analyals  purposes, 

Thin  paper  presents  a  typical  analysis  uf  service  practice  firing 
remits  and  Indicates  th*  significance  of  these  results  in  the  Surveillance 
Program,  An  example  of  the  evaluation  of  the  annual  service  prattles 
firings  for  ths  Honest  John  Rocket  will  be  presented 

DISCUSSION.  In  this  evaluation  1*4  firings  of  Rocket  76kMMi 
Mil  Series,  conduct'd  for  tr-nip  training  and  other  purposes  by  both 
United  Sutei<  <uid  N/vTO  tiring  unite  have  been  considered.  The  purpeso 
of  this  study  was  to  investlgat-:  the  overall  accuracy  performance  of  the 
M.'il  rocket  system  when  fired  by  troop  unit  ft  and  to  establish  if  there  is 
any  indication  of  a  deterioration  of  this  acruracy  performance  with 
lrrrcaslnc  age  of  the  M*  series  rockst  motors  of  these  MJ1  s«<T*n  rockets. 

In  investigating  the  effect  of  age  on  accuracy  pcriVvmanr*,  two  possible 
type*  uf  changes  wars  considered!  first,  changed  in  the  canter  of  Ir  pact 
(CL)  of  t'-e  rackets  c.v  a  function  of  age  and  second,  changer  in  the  dis¬ 
persion  of  the  rockets  about  the  C,l.  as  a  function  of  c,ge  For  example, 
if  the  total  impulse  of  the  rockst  motor  which  in  a  measurement  of  the 
total  thrust  of  the  rocket  decreased  with  increasing  age,  then  the  mean 
range  of  the  rocket  should  also  decrease  with  Increasing  age.  It  Is  also 
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possible  that  *•  the  age  of  the  rocket  motor  increase*,  it*  performance 
could  become  more  erratic  thus  increasing  the  range  dispersion  of  the 
Honest  John  Rocket,  Although  the  effect  of  age  on  both  range  and 
deflection  mtss-diatances  was  investigated,  in  general  it  is  expected  that 
deterioration  of  rocket  motor  performance  will  be  reflected  primarily  in 
changes  in  the  range  performance  of  the  M31  series  rocket.  In  order  to 
investigate  the  affect  of  age,  simple  and  multiple  linear  regression 
technique*  and  analysis  of  variance  techniques  were  used. 

In  investigating  the  effect  of  age  on  msan  range  and  deflection  perform* 
ance,  all  firings  were  grouped  according  to  the  type  of  burst  and  type  of 
launcher  system  employed,  Separate  analyser  were  carried  out  for  each 
of  thesn  groups  for  both  rang*  and  deflection  miss-distances.  Th*  re¬ 
gression  model  used  in  these  analyee*  assumes  range  and  deflactlon  mlas- 
dlstances  for  each  firing  are  linear  functions  of  tha  age  of  the  rocket  motor 
and  the  launcher  to  target  range.  The  launcher  to  target  range  was  Included 
in  th#  regression  model  sine*  any  age  affect*  would  probably  be  range 
dependant. 

In  the  division  of  the  rocket*  according  to  typa  of  burst  and  typs  of 
Launcher  system,  the  majority  of  the  rockets  (84B  of  the  924  considered) 
fell  into  thrse  main  groups,  These  were  firings  with  the  M289  launcher 
lor  iir  burst  (211  sockets  fired),  M386  launcher  for  air  burst  (473  rocket* 
fircl)  and  the  M289  launcher  for  ground  burst  (164  rockets  fired).  In 
addition,  there  were  49  rockets  fired  with  ths  M33  launcher  for  air  burst, 

21  firings  with  the  M386  launcher  for  ground  burst  and  6  firings  with  M33 
launcher  for  ground  burst,  The  regression  analyses  which  were  carried 
out  with  all  these  group*  ex-.spt  for  tha  firings  with  ths  M33  launcher  for 
ground  burst  due  to  only  six  rockets  Heins  fired  in  this  group  indicated 
that  after  7  1/2  years  of  shelf  life  th- re  appears  to  be  a  significant  age 
effect.  This  is  indicated  by  a  signlficsnt  age  effect  (.  01  level)  in  tue 
analysis  of  rang*  miss-distance  in  the  M386  launcher  firings  for  air 
burst.  Although  none  of  the  other  groups  indicated  a  significant  age 
effect  at  the  .  01  level,  the  other  two  Large  groups  of  rockets  were  near 
significance  at  this  level.  For  example,  for  the  group  containing  rockrts 
fired  with  the  M289  launcher  for  ground  burst  t  »  -2,  25  which  is  r„i, ' 
significant  at  the  .  Cl  level  a*  t  ■  2,  61  but  is  significant  at  the  .  05 

level  where  t  3  1*  97.  With  the  group  containing  rockets  fired  from 

M289  launcher  lor  air  buret  t  «  -1.95  which  is  just  short  of  tho  .  05 
level  of  significance,  Thus,  it  appear*  that  there  is  an  indication  of  a 


Design  of  Experiment* 


625 


decrease  in  the  mean  range  performance  a*  a  function  of  age  of  the 
rocket  motor  of  the  M31  rocket,  No  significant  (.  01  level)  age 
affect  was  found  in  the  analysis  of  deflection  mis* -distance  a*  a 
function  of  age. 

In  the  analysis  of  the  M3&6  launcher  firings  tor  air  burst  in  which 
a  significant  (.  01  level)  age  effect  was  indicated,  a  significant  (.  01 
level)  launcher  to  target  range  effect  was  also  noted,  To  further 
investigate  these  significant  results  the  rockets  within  this  group  were 
further  subdivided  into  four  launcher-to -target  rang*  groups,  These 
group*  wars  rockets  with  a  iauncher-to-target  range  of<10,000 
maters,  10, 000  to  14, 000  meters,  14, 001  to  18,  000  meters  and 
>  18,000  meters,  A  regression  analysis  of  range  miss-distance 
as  a  linear  function  of  rocket  motor  age  was  performed  on  each  of  the 
four  launcher-to-target  range  groups.  The  regression  analysis  for  the 
10,  000  to  14,000  meter  group  which  contained  the  largest  numbsr  of 
rockets  fired  indicated  e  significant  (.  01  level)  ega  affect,  A  scatter 
diagram  of  the  range  miss-distance  for  the  four  Iauncher-to-target 
range  groups  is  presented  on  Figures  l  thru  4.  Figures  1,  3  and  4 
indicate  tl.e  range  miss-distances  versus  the  M6  series  motor  age 
for  tne  three  range  groups,  <.10,000  meters,  14,001  to  18,000 
meters  end>18,000  mstsrs  in  which  no  significant  (.01  lsvsl)  age 
effect  is  Indicated.  Since  there  was  no  significant  ege  offset  for 
these  three  groups  the  mean  miss-distance  for  each  group  is  also 
indicated  on  these  figures,  A  scatter  diagram  of  the  range  miss- 
distances  versus  the  M6  series  motor  age  for  the  rocket*  fired  in 
the  10,  000  to  14, 000  range  group  is  presented  on  Figure  2.  As 
previously  indicated,  a  significant  (.01  level)  age  effect  wee  noted 
in  this  group,  Indicated  or  this  figure  ie  the  regression  line  derived 
from  the  regression  analvi'r  performed  with  this  group.  This  regres¬ 
sion  line  indicate*  that  the  average  range  of  each  rockai  for  th*  period 
covered  has  been  decreasing  at  the  rate  of  18,  b  mntere/yaar  such 
that  the  average  range  miss -distance  for  7  l/2  year  old  motors  was 
80  meters  (80  meters  short  of  the  target). 

As  previously  indicated,  in  addition  to  an  age  effect  with  the  M*8& 
launcher  firings  for  air  buret,  a  significant  iauncher-to-target  range 
effect  wav  noted,  Aa  Indicated  on  Figures  1 ,  3  and  4  the  average  range 
misa-diitance  was  -50  meters  for  th*  <.10,000  meter  range  group, 

+  27  meters  for  the  14, 001  to  18,  000  meter  group  and  +  104  meters  for  ths 
>18,000  meter  group.  These  results  seem  to  indicate  that  for  the  shorter 
range  targets  the  rockets  are  falling  short  of  ths  target  and  as  the  target 
range  increases  ‘.he  rocketii  begin  going  over  ths  target  so  that  at  the  long 
ranges  (>  1 8,  000  meters)  the  rockets  ere  tailing  on  the  average  104  meters 
beyond  the  targvi.  This  occurrence  is  probably  due  to  a  small  bias  in  the 
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firing  tabln.  Table  1  contain*  a  nummary  ol  the  reported  mi  ••-distance* 
for  the  M386  launcher  firing*  for  air  bunt.  The**  flringa  are  grouped 
according  to  age  and  according  to  tha  rang*  group*  pravlounly  mentioned. 
On  Table  1,  It  i*  noted  that  for  rocket*  fired  at  rang*t<.  10,  000  meter* 
the  reported  mean  ml**-diitance*  for  all  age*  are  negative  except  the 
6  to  7  year  old  rocket*,  Thi*  1*  alio  Indicated  In  th*  10, 000  to  14,  000 
mater  group  except  in  thi*  group  the  3  to  4  year  old  rocket*  mean  ml**- 
diatance*  are  positive,  Where**  in  tha  14,001  to  18,000  meter  group  all 
the  mean  mli*-dl*t*nca*  ure  pueitiv*  with  the  exception  of  the  4  to  5 
year  old  rocket*,  Similarly  In  the  >  18,  000  meter  group  all  tha  mean 
miss-distance*  are  positive. 

In  inveetigating  the  poadhle  effect  of  age  on  range  end  deflection  dit- 
pereion  regreeiion  technique*  were  *Uo  employed.  In  order  to  remove 
the  dependence  ef  th*  variance  of  cell*  on  the  true  population  variance  of 
the  cell  a  logarithmic  transformation  of  th*  range  and  deflection  eampla 
variance  was  mad*  prior  to  carrying  out  th*  r*gr*  salon  analyse!.  Due  to 
th*  limited  number  of  ground  impact  firing*,  the**  analyaa*  were  limited 
to  th*  sir  buret  round*.  Separate  analyse*  war*  carried  out  for  such 
calendar  year1*  firings  with  each  launcher  e*  well  e*  a  combined  analysis 
over  ail  calendar  years  for  each  launcher.  Tl.ee*  analyaa*  disclosed  no 
significant  (,  01  level)  age  effect  for  either  rang*  or  defection  dispersion. 

Table*  2  and  3  present  a  summary  of  tha  mean*  and  standard  deviation* 
of  rang*  (R,  Sp) ,  deflection  (E;  S^)  end  height  of  buret  (fl,  S^)  misa- 

dlfftsnce*!  Table  2  lummarlee*  those  rounds  fired  for  air  burst  and  Table 
3  summarise*  those  rounds  fired  for  ground  Impact,  Each  table  cortaine 
a  separate  summary  for  th*  firing*  conducted  during  each  calendar  year 
and  a  combined  summary  over  *11  firings,  Th*  firing*  far  each  calender 
yoat  are  grouped  according  to  th*  age  of  the  M6  eerie*  motor  in  one-year 
intervals,  These  grouping*  are  further  subdivided  according  to  th*  type 
of  launcher  system  employed,  l.e,,  M289,  M3B6  end  M33.  Prior  to 
computing  th*  sample  mean*  *nd  standard  deviations  in  these  tables,  th* 
consistency  of  the  data  was  investigated  by  testing  the  extreme  value* 
using  l  criterion  for  outlying  observations,  Any  outlying  value*  (at  th* 

.01  level  of  significance)  we ru  not  need  in  computing  the  mean*  and 
standard  deviations  of  th*  mi** -distance!,  but  were  tabulated  individually 
as  footnotes  to  the  table*. 
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SUMMARY,  In  summarising,  the  results  of  theie  analyses  have 
indicated  that  there  apptara  to  he  a  significant  age  effect  on  the  functlon- 
ing  reeults  of  the  M3I  Seriee  Honeit  John  Rocket.  Thie  1*  indicated  by 
the  regreeilon  analyeee  carried  out  with  the  M3B6  launcher  flrlnge  for 
air  burat  and  with  the  M2B9  launcher  firing*  for  ground  and  air  buret. 

With  the  M386  launchor  firing*  for  air  burat,  the  average  range  appear*  to 
be  decreasing  at  the  rate  of  IB.  6  meters/yr  such  that  for  7  1/2  year  old 
motor*  the  average  range  mi**-di*tance  1*  -BO  meters.  It  ahould  be 
polnieu  out  that  the  decreaia  in  range  of  IB.  6  mater*/yr  i*  considered  a* 
the  rate  of  decrease  for  rochets  1  to  7  1/2  year*  old  and  should  not  be 
construed  a*  indicative  of  a  rate  of  decrease  for  rochets  older  than 
7  1/2  year*,  Extrapolation  of  thie  rate  of  decrease  to  older  motor*  may 
be  hasardou*  a*  the  rata  may  change  a*  the  motors  age.  It  might  also 
be  pointed  out  that  the  decrease  in  range  observed  for  older  motors 
appear*  to  be  substantiated  by  rocket  motor  static  tests.  It  wan  also 
noted  with  this  group  of  firings  that  there  was  a  significant  launcher-to* 
target  range  effect.  This  affect  appears  to  indicate  that  the  shorter  range 
rochets  (<.  14,000  meters)  ars  impacting  short  of  the  target  while  the 
longer  range  rockets  ('-14,000  meters)  are  Impacting  beyond  the  target. 
This  appear*  to  be  due  to  a  small  bias  in  the  firing  table. 

CONCLUSION,  in  conclusion,  we  might  consider  the  affect  of  the 
significant  age  and  launcher  to  target  range  effect  on  the  currant  statu* 
oi  the  M31  stockpile,  First,  it  should  be  pointed  out  that  the  bulk  of 
the  stockpile  consists  of  rockets  that;  vary  in  ages  from  3  to  6  years, 

Fr,r  rocket*  of  these  ages,  with  the  ags  effect  Indicated  by  the  M3B6 
launcher  firing*  for  air  burst  an  average  miss-distance  of  0  to  -SO 
meters  due  to  age  may  wc  expected.  Thus,  the  bulk  of  the  stockpile 
does  not  si  yet  appear  to  be  adversely  affected  by  this  age  effect, 

If  at  some  later  date,  it  is  determined  that  the  age  effect  is 
reducing  the  effectiveness  of  the  Honest  John  Rocket  System  It  would 
be  poseibl*  to  correct  for  this  sgs  effect  by  establishing  a  propellant 
weight  correction  factor  to  bn  used  with  these  rocket*. 

In  relation  to  future  analyses,  tha  Surveillance  Qroup  Is  continually 
reviewing  all  service  practice  firings  for  inclusion  in  theso  analyser. 

As  mentioned  earlier,  the  evaluation  of  service  practice  firing*  if 
only  a  part  of  an  integrated  program  which  has  besn  initiated  to  deter¬ 
mine  the  reliability  and  ballistic  characteristic*  of  the  artillery  missile 
stockpile. 


S  Series  *!otor  A?,e  ('•'onths) 


Rocket,  762MM:  *131  Series  -  M386  Launcher 

Range-Miss  Distance  Vs.  M6  Series  Motor  Ape 
Range  14.001  to  18.000  Meters 
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THE  MEASUREMENT  OF  THE  MORALE  AND  SUPPRESSIVE 


EFFECTS  OF  WEAPONS* 

D.  H.  Chaddock 

C.B.F..,  M.Sc.,  M.I.  Much  E, 


In  the  teat  decade  considerable  progress  haa  been  made  in  aaotsalng  at 
an  early  atage  the  probable  performance  of  propoaed  weapon  eyetema  In 
terma  of  their  chance  of  a  hit,  and  giver,  a  hit,  the  extent  of  tha  damage  they 
will  inflict.  Information  of  thia  kind  ia  frequently  need  to  compare  one 
weapon  ayatem  with  another  or  to  decide  whether  a  particular  project  ia 
worth  purauing  further.  The  methods  uaed  are  mainly  atatiatieal  and  are 
baaed  upon  meaauremanta  made  during  controlled  trlele,  nr  extrapolation 
from  aueh  data. 

In  ao  far  aa  phyaidal  damage  to  equipment  or  men  la  concerned  the 
baalc  data,  although  not  all  that  could  ba  da  aired,  ia  sufficient  for  a 
reaaonably  aatiafactory  aaaaaement  to  bo  made  of  the  Derail  effectiveness 
of  the  weapon  system,  T;i  caaea  where  thia  is  high  (for  example,  over  50%) 
it  la  not  porhapa  an  unreasonable  assumption  that  the  ayatem  must  posaesa 
'ran  value,  and  other  things  being  equal,  should  receive  further  development, 
.in  other  eaeea,  due  either  to  the  uxtreme  difficulty  oi  the  teak  that  hae  been 
proposed  or  limltatione  imposed  on  the  aeeeeament  (for  example,  in  weight 
and  alee)  it  ia  not  possible  to  snow  that  a  proposed  system  would  have  more 
than  a  vary  amall  effectiveness,  perhaps  of  the  order  of  a  few  percent.  In 
three  case*  it  ia  natural  to  reject  the  proposal  aa  valueless  for  military 
purposes. 

A  decision  of  thia  nort,  while  valid  in  relation  to  tho  material  damage 
which  la  being  discussed,  admittedly  completely  ignores  any  other 
which  tha  uaa  of  tha  weapon  might  produce.  Typical  examples,  wall 
known  to  Servica  personnel  are  -- 

1.  "Raising  the  morale"  from  the  point  of  view  of  the  man  firing 
the  Weapon, 

i.  "Deterring  the  attacker"  from  the  point  of  vio'.v  of  the  man  being 

fired  at. 

3.  "Keeping  their  heads  down"  from  both  points  of  view. 

A  brief  examination  of  three  typical  caaoe  might  serve  to  Illustrate 
the  type  of  situation  in  which,  although  the  probability  of  m-ierlal  damage 

*  This  paper  by  D,  H,  Chaddock,  Director  of  Artillery  Research  and 
Development,  The  War  Offica,  United  Kingdom  was  presented  at  the 
conference  by  G,  F.  Komlosy. 
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might  ba  amall,  the  "Morale"  value  of  delivering  an  attack  could  be  quite 
high. 

INFANTRY  ANTI-TANK  WEAPONS.  BecmiLc  of  limitations  In  weight 
and  aUe  muat  infantry  anti-tank  weapons ,  auch  aa  the  Basooku,  the 
Panarrfauat  and  the  Carl  Gustav,  deliver  their  attack  on  impact  by  .neane 
of  a  hollow  charge  warhead.  The  hollow  charge  produrea  a  high  velocity 
Jet  which,  although  it  can  penetrate  a  conriderablo  thicknaaa  of  ateel, 
produce  a  only  a  relatively  amall  hole.  Inside  the  target  the  remnanta  of 
the  Jet  and  the  fragment*  from  the  hole  are  confined  to  a  relatively  narrow 
com  having  an  axia  in  prolongation  of  the  light  path.  Within  the  cone  the 
fragmenta  have  considerable  damaging  power,  which  can  he  readily 
assessed  In  term*  of  material  and  men.  Outride  the  cone  the  reeulte  appear 
to  bo  very  disappointing,  even  whan  anoathetlscd  animala  are  subjected 
to  attack. 

Therefore,  scoring  only  on  the  mensurable  physical  damage,  asaaes- 
manta  lead  to  a  rather  low  probability  of  "killing"  a  tank  by  this  means. 
However,  such  cufaeaements  arc  not.  universally  acceptable,  French 
experts,  for  example,  consider  that,  when  a  tank  is  penetrated  by  a  hollow 
charge,  the  crew  will  be  ho  demoralised  that  they  will  be  unable  to  continue 
to  fipht.  Direct  evidence  of  this  la  hard  to  come  by.  Some  British  crewa 
report  boing  unaware  of  having  been  hit  until  after  the  battle  when  they  have 
discovered  a  hole  in  their  tank, 

ATTACK  OF  LOW  FLYINQ  AIRCRAFT.  Ever  since  alrcrait  flew  in 
battle  <i,«re  has  been  a  problem  of  attacking  them  from  ground  baaed 
weapon",  Owing  to  the  fundamental  difficulties  of  detecting.,  sighting  and 
aiming  at  a  moving  target,  and  in  spite  of  a  great  deal  of  ingenuity  expended 
in  the  development  of  sighting  systems  and  weapons,  the  problem  has  never 
beer,  nnvlefactorily  eolved,  Current  maeeesmenta,  for  example,  show  id.it 
for  a  r.imple  20mm  machine  gun,  eye  sighted  and  manually  laid,  the  chancea 
of  successfully  engaging  a  low-flying  target  are  very  amall  even  under 
favorable  cirrumstenr""  Technically  eophistlcated  notations  involving  the 
usu  of  radar  for  early  warning,  range  finding,  etc.  lead  to  weapon  systems 
too  complicated  to  envisage  as  an  all -arms  anti-aircraft  weapon. 

The  current  range  of  armoured  fighting  vehicles  are  therefore  being 
produced  without  any  meana  of  defending  thdmeelves  from  low  flying  air¬ 
craft,  on  the  grounds  that  even  if  they  had  a  weapon  It  would  be  Ineffective. 
This  situation,  although  accepted,  is  causing  soma  disquiet.  Users  with 
battle  experience  remember  the  times  whun  they  fired  their  weapons  at 
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inquisitive  enemy  aircraft  end  believe  they  aeUrrcd  them  from  carrying 
out  the  purpose  of  their  sortie  by  doing  so.  The  reaction  of  aircraft 
pilots  to  being  fired  at  no  doubt  veiled,  but  the  general  impression  seems 
to  be  thet  elthough  "flak"  from  heavy  anti-aircraft  guns  wee  frightening  to 
unseasoned  crews,  end  e  nuisance,  it  was  never  a  real  deterrent.  Small 
arms  fired  from  the  ground  seam*  to  be  ignored  by  most  pilots.  This 
might  not  be  the  case,  however,  with  relatively  unprotected  and  slov/» 
moving  helicopters. 

ARTILLERY  SUPPORTING  FIRE.  On*  of  the  classical  roles  of 
Artillery  la  to  provide  supporting  fire  for  other  arms  in  both  defence  and 
attack.  In  general  this  involves  bringing  indirect  fire  to  bear  upon  an  area 
target.  Except  in  certain  specific  cases,  for  example  counter  battery  fire, 
no  attempt  is  made  to  engage  Individual  target*  within  the  area,  the  volume, 
rate,  and  weight  of  fire  being  judged  as  likely  to  produce  some  casualties 
and  general  disruption  of  the  enemy's  activities,  In  the  past,  when  Infantry 
could  only  move  by  exposing  themselves,  they  were  vulnerable  to  attack  by 
high  explosive  shell  fragments,  particularly  when  VT  fuses  were  used,  and 
ths  efficacy  of  this  eort  of  fire  is  unquestioned,  Its  very  efficacy  has  ltad 
to  the  development  of  the  armoured  personnel  carrier  and  self  propelled 
and  armoured  guns,  in  whose  military  specification  there  is  a  requirement 
that  they  shall  protact  thsir  occupants  from  the  offsets  of  near  burst,  high 
oxploalva  shslls.  Assessment  therefore  of  th»  material  damage  likely  to 
be  inflicted  on  a  highly  mechanised  army,  such  as  the  Russian  Army,  by 
indirect  fire  with  conventional  weapons  Is  depre  a  singly  low,  The  user, 
Sowevsr,  is  not  prepared  to  accept  the  conclusion  that  hit  fir*  is  tharsfore 
.ineffective  and  point*  to  the  morale  effect  of  subjecting  the  troops  to  fire 
of  any  sort.  In  tM*  it  is  a  widely  held  belief  that  a  single  large  exploelon 
very  much  more  effective  than  the  same  weight  of  fire  delivered  in 
a  number  oi  smaller  explosions --the  philosophy  of  the  "hig  crump", 

The bc  are  typical  caeea,  and  in  them  and  other*  It  Is  quite  obvious 
that  experienced  users  believe  that  their  weapons  can  and  dc  produce  an 
effect  over  and  above  the  material  damage  that  can  be  observed.  So  far, 
no  mean*  of  measurement  or  aseesement  has  been  found  for  thia  so-tU*d 
deterrent  or  neutralisation  effect,  Assuming  that  such  an  effect  decs  in 
fact  exist,  if  it  is  small  then  no  graat  harm  will  be  done  If,  a*  now, 
weapons  are  assessed  solely  on  the  material  damage  they  can  Inflict.  If, 
on  the  other  hand,  the  deterrent  effect  Is  large,  then  it  might  materially 
affect  the  choice  of  weapon  system  and  armament  with  which  the  Forces 
are  provided, 
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To  Investigate  and  manure  these  effects  li  obviously  going  to  be 
difficult,  if  not  impoaaible  by  direct  experiment,  because  unleea  there  ie 
real  danger  the  subject1*  reaction  may  not  be  representative.  Simulated 
battle  as  provided  In  "Battle  Indoctrination1'  course*  is  unlikely  to  be  a 
satisfactory  substitute  for  the  real  thing  since  however  well  done  it  is, 
the  participants  retain  confidence  that  their  instructors  and  safety  officers 
would  allow  very  little  real  danger.  An  analysie  of  past  battle  experience 
might  produce  some  uaahla  data,  although  debriefing  now  on  the  memory 
of  events  more  than  a  decade  age  could  be  very  unreliable,  A  possibility 
llial  suggest*  Itself  Is  i:he  use  of  experiments  using  drugs  or  hypnosis  to 
pursued*  the  participants  that  they  really  are  in  danger,  and  that,  for 
example,  the  simulated  enemy  1*  really  shooting  to  kill,  then  their 
reactions  to  a  "real"  situation  can  perhaps  be  assessed. 

Personally,  as  a  layman,  I  believe  that  troops  in  battle  very  rapidly 
make  their  own  assessment  of  the  lethality,  imagined  or  real,  of  the 
situation  in  which  they  find  themselves,  and  act  accordingly.  Below  a 
certain  threshold  the  risk  is  acceptable  and  they  proceed,  if  with  some 
caution,  in  the  normal  manner.  Above  this  threshold  the  risk  is  intoler¬ 
able  and  action  is  taken  to  avoid  It,  This  suggests  that  a  clue  might  be 
found  in  other  dangerous  •ntlvitles,  such  as  mountain  climbing,  working  on 
high  buildings,  or  even  crossing  the  road,  A  city  dweller  will  cross  the 
rcutl  "jay-walking"  in  the  fact  of  oncoming  traffic  because  he  believes  that 
the  risk  is  negligible.  A  country-man  or  elderly  person  will  not  accept 
the  risk  and  either  crosses  at  lights  or  wait*  until  the  road  is  cloar.  In 
the  construction  of  high  building*  normal  rates  are  paid  to  worker*  below 
a  certain  height,  above  it  they  receive  a  bonus  of  danger  money.  If 
condition*  are  bad  with  high  winds,  ice  and  snow  etc. ,  all  work  ceases, 
the  risk  is  intolerable  to  employ**  and  employee  alike.  Many  similar 
examples  could  be  drawn  from  other  activities!  in  all  of  them  we  find  the 
acceptance  of  a  certain  degree  of  risk  but  a  threshold  above  which  uic 
risk  l*  intolerable,  If  a  threshold  value  really  doea  axist  and  can  be 
fuund,  then  U  -Would  be  possible  to  relate  it  to  the  probability  ol  materi¬ 
al  or  physical  damage. 

Ac  to  what  the  threshold  value  might  turn  out  to  be,  I  believe  thn 
it  will  be  found  to  be  much  smaller  than  generally  supposed.  The  Romans, 
who  knew  a  great  deal  about  the  conquest  and  subjugation  of  foreign  countries, 
gave  us  the  word  "decimate"  which  according  to  the  dictionary  mean*  to 
kill  a  tenth.  1  believe  that  now,  as  then,  if  one  tenth  of  a  group  of  Infantry, 
tanks,  aeroplanes  or  whst  have  you  are  killed  and  seen  to  be  killed,  the 
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morale  of  the  survivors  will  deteriorate  very  rapidly.  This  la  1a  sharp 
contrast  to  much  current  military  thinking  and  might  well  revolutionise 
some  of  our  current  military  requirements,  and  make  much  more 
attractive  proposals  for  weapons  systems  that  would  now  be  throv/n 
Into  the  waste  paper  basket.  For  this  reason  1  think  It  Important  to 
direct  scientific  thinking  and  research  into  this  area.  1  have  given  sn 
account  of  some  of  the  difficulties  that  will  be  mat  with,  and  made  e 
few  suggeiitions  regarding  possible  avenues  of  approach,  1  would  now 
be  most  Interested  to  hear  the  views  of  any  member  of  the  audience  on 
this  subject,  particularly  with  regard  to  ».ny  actual  experimentation  that 
may  have  been  performed. 

COMMENTS  ON  THE  PAPER  BY  D.  H,  CHADDOCK 
Clinical  Saasiun  C 


As  a  member  of  thle  panel,  Boyd  Harohbarger  suggested  that  under 
the  topic  "The  Measurement  of  the  Morele  and  Suppreealve  Effect*  of 
Weapons"  many  discoveries  in  this  area  could  possibly  be  derived  by 
having  psychological  studies  madt  u t  the  reactions  of  the  personnel 
employed  at  munition  Installations  and  manufacturers  In  which  they  have 
had  explosions.  For  instance,  at  the  Allegheny  Ballistic  Laboratories 
a  rvroup  of  psychologists  might  have  been  able  to  evaluate  many  of  tha 
character lalii »  that  aitecl  morale  bv  observing  the  action  of  the 
neraonnel  alter  a  major  explosion  that  happened  there  about  s  veer 
ago.  Psychological  designs  could  be  set  up  for  activities  at  various 
areas  In  dangerous  sections  to  study  human  reactions  to  emergencies. 
These  designs  should  be  set  up  prior  to  liny  accidents  for  various 
plants  end  Installations,  and  should  be  ready  to  be  executed  immediately 
upon  a  catastrophe. 


SOME  SMALL  SAMPLE  THEORY  FOR  NONLINEAR 
REGRESSION  ESTIMATION 


H.  O,  Hartley 

Institute  of  Statistics,  Agricultural  and  Mechanical 
Collage  of  Texaa,  College  Station,  Text* 


NOTE ,  The  paper  read  at  the  Ninth  Conference  by  H.  C,  Hartley  under 
the  title,  "Non-Linsur  Estimation"  pi  aaeuiad  results,  both  published  and 
unpublished,  on  the  general  topic  of  leaat  squares  eetimation  of  the  para-' 
meter*  on  non-linaar  regression  laws,  For  material  already  publiahed 
or  about  to  be  publiahed  [in  thU  areaj  reference  la  made  tot 

H,  O,  Hartley  (1961),  "The  Modified  Oausi-Newton  Method 
for  the  fitting  of  Non-Linear  Ragreaaian  Function*  by 
Leaat  Square*, 11  Technometric*  J,  269. 

H,  O,  Hartley,  "Exact  Confidence  Regions  for  the  Parameter* 
in  Non-Linear  Ragreaaian  Law*, 11  (shortly  to  be  published 
In  Plometrlka. ) 

H,  0.  Hartley  and  A,  Booker,  "Non-Linear  Leaat  Squares 
Eetimation",  Accepted  for  publication  to  the  Annals  of 
Mathematical  Statlatlca , 

Thu  note  reproduced  below  represent*  a*  yet  unpublished  material, 

1.  INTROOUGTION  ANP  SUMMARY,  Wu  are  gtVc,j.  *  set  of  N 
reeponsee  vi  which  have  arisen  frorr.  e  nonlinear  regrnaalon  model 

(l)  yt  ■  ft*.,  0)  +  *tt  t  »  i,  i . n 

Uure  xt  dsnote*  the  t**1  "fixed"  input  vector  of  k  elements  giving 
rtM  to  y  ,  whilst  d  is  an  m-element  unknown  parameter  vector  with 
elements  0i  and  the  e.  are  a  set  of  N  independent  error  roeidualt, 
from  N(0,<r  )  where  er  c  will  be  assumed  either  known  ur  unknown  ae 
indicated  in  the  various  sections,  The  expectations  of  the  y{  are  therefore 
the  k+m  variable  functions  f(jc,  0)  whicn  will  be  assumed  to  satisfy 
certain  regularity  conditions.  The  problem  is  to  estimate  0  notably 
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by  "losst  aq'ibiui"  which  in  tha  present  raea  ii  identical  with  "maximum 
likelihood".  Whilat  computational  procaduraa  to  obtain  tha  nonlinear 
leaat  squares  estimators  and  their  large  sample  properties  have  received 
some  consideration  in  the  literature,  we  are  in  this  note  concerned  with 
deriving  soma  small  sample  results  for  the  above  estimation  problem. 

2  A  THEOREM  ON  THE  SUFFICIENCY  IN  NONLINEAR  REPRESSION 
ESTIMATIN',  In  what  to llows  wa  shall  be  predom in antiy  using  vector 
notation  and  shall  denote  by  y,  f(0),  e  and  &  vectors  having  elements  y{, 
f(x  ,  0),  e(  nnri  9  respectively.  The  first  three  of  these  are  N-voetors, 
the  last  an  m«vector.  The  N  x  k  matrix  of  input  values  (x  )  will  not  be 
specifically  referred  to  in  this  section  and  these  matrix  elements  merely 
enter  as  fixed  arguments  into  the  functions  f(x  ,  9),  The  nonlinear  model 
(1)  can  then  be  written  as 


(2)  y  «  t(«)  +  e 

and  the  Joint  distribution  of  y:  l.e.  the  sample  likelihood  as 

(3)  L(6,  y)  .  U.r«rV*N  axp  -±j-  (y'V  *  */*(•>  +  <'(•)  «(•>] 

whdi't  the  1  denotes  transposition  of  column  vectors  and  where  ws  assume 
that,  for  tha  time  being,  r*  is  known.  Adopting  classical*  definitions  o» 
etatMlcal  sufficiency  we  say  the  L(  6,  y)"  admits  a  sat  of  p>statistlcs 
s(y)  jointly  suiiiclent  for  the  estimation  of  6M  if 

(4)  Me,  y)  s  r'S(y),  6)  H(y) 

where  the  p-vector  s(y)  has  as  its  elements  tho  p-statlstics  *^(y>  which 
a.re  mathematical  N. variable  functions  of  the  N  elements  v, ,  ,  .  y, ,  oi 
the  sample  vector  y  so  that  F  is  a  p  +  m  variable  function  of  the  ulemnnta 


Distribution  functions  ami  statistics  ure  here  assumed  to  bn  at  least 
piecewise  continuous  functions  of  their  argments,  i.  e,  foe  all  y  and  »U 
9  in  a  convex  n  dimensional  space  ft. 
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ot  *(y)  and  6,  whilst  H(y)  ii  an  N-variabl*  function  of  y.  Comparing 

(4)  with  (.1)  wo  not*  that  the  condition  for  joint  sufficiency  can  be  written 

a* 

(5)  y'f(»)  •**•(«)  r(»)  •  J(*(y),  9) 

where  wj  have  introduced  the  p  *■  m  variable  function  3  by  the  equation 

(6)  F(e(y),  9)  *  exp  J(a(y),  0)J  _ 

We  now  state  the  following  theorem  which,  under  certain  regularity  condl* 
Lions,  is  more  or  less  obvicust  - 

Thoore m  li- 

Fnr  the  existence  of  a  swt  of  o  statistics  which  acfjolatllt Jtuff.lfligfU 
for  a  «et  of  p  oarameter  functions  gj(»)  (j  «  1,  2.  ...  b)  ft  is  sufficient 

and,  under  regularity  condition  C,  nsceirarv  that 

(7)  f(0f  «  U  g(0) 

where  H  i*_an  n  x  p  matrix  of  fixed  numbers  und  g(0)  i!»  a  p«vector  with 
fti.aa»#m*  gj(«)  - 

Ths  regularity  condition  C  mentioned  in  Theorem  i  is  as  follow*, * 
Tlitre  exists  at  least  ous  N-vector  y(*ay  y  *  h)  such  that  all  Sj(v)  ha.e 
first  partial  differ  sniibj.s  y  ■  q  and  the  function  3  lieflneh  by  (6) 
ha*  first  partial  differentials  with  regard  to  the  s.  at  T  u  *(',))  tor  all 
9  in  XL.  J 

Proof  of  Theorsm  i 

Th*  sufficiency  oi  (7)  is  trivial.  We  hav#  from  (7) 

2y'  f(0)  -  2y'  V  g(0)  «  2*'  (9)  U' y 

f*  CO)  f(B)  -  g'(9)  U'  Ug(0) 

Substituting  (0)  and  (9)  in  (3)  we  have  the  form  (4)  for  the  parameter 


(8) 

and 

<!»> 
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functions  g(B)  and  the  p  statistics  e,(y)  which  arc  the  elements  of 
the  p- vector  ' 

(10)  e(y)  -  U1  y  , 

To  prove  that  (7)  is  necessary  we  are  given  a  likelihood  function  which 
H«tisfiu«  (4),  (3)  and  (6)  and  which  we  restrict  to  satisfy  the  regularity 
condition  C,  In  view  of  the  Utter  condition  wu  are  able  to  apply  to  the 
function  tf(*(y).  8)  the  standard  differentiation  rules  at  the  paint  y  «  r| 
and  s(v)  «  s(ii)  «  T,  We  therefore  obtain  by  differentiating  (3)  with 
regard  to  y  the  rotation 

V 

(11)  f(x ,  e)  »  £ 

1  fl,j  8yt 

or  the  vector  form  (7)  if  we  define  the  N  n  p  matrix  V  by 

(12)  U  - 

Byt 

a..d  the  p-vector  g(8)  by 

<>»  .<•>  ■  -“is^  ■ 

Theorem  1  shows  that  we  r«nnot  Hxpect  to  obtain  a  sufficient  set  of 
sW'.tlstics  unless  the  regression  function  f(8)  is  of  the  "eseentisUy  linear" 
form  By  essentially  linear  we  mean  that  it  ii  linear  in  a  eat  ot  p  paramet 
function*  Cj(8),  j  ■'1,2 . p, 

The  eatimabillty  of  the  linear  parameters  g  (8l  depends  on  the  rank  of 
the  matrix  U,  in  accordance  with  linear  estimation  theory.  The  sprcUl 
cc.ko  of  p  »  m  and  reek  (II)  »  m  i*  of  interest  as  the  number  of  g  (8)  la 
thus  equal  to  the  number  of  the  original  parameters  8.,  In  such*  situation 
it  may  be  asked  under  which  circumstances  the  mapping  of  the  It  specs 
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of  0.  to  the  l  '-apace  of  the  g.(0)  la  a  one-to-one  mapping.  WhiUt  the 
clairiicHl  condition  of  a  full  rank  Jacobian  only  allure*  thia  one-to-one 
mapping  "in  the  email"  we  prove  the  following  unique-mapping  theorem 
"in  the  large'l  • 

Theorem  2i* 

If  for  a  lit  of  m  function*  gj(8)  depending  on  m  variable*  «j  w.e 

a»*ume  continuity  of  the  flr*t  partial  d*rlvatlv.**  and  d*flnltan**a-aLth» 
quadratic  form, 

m  . 

(14)  S  i  u,  u.  0  for  u  ♦  0 

l,j*l  Bdt  1  J  < 

for  all  6  in  the  convex  fi.«aac*  then  the  mapping  ja.f 0  ta  F  b# 

(13)  V  -  ?.(») 

J  J 

1*  a  one-to-one  mapping. 

Proofi- 

We  have  to  ihow  that  two  different  9  vector*  alway*  generate  two  _ 
different  o -vectors,  Suppose  that,  on  the  contrary,  for  •+  and  in  i'i* 
we  find 

(16)  g(^1')  «  g(k>')  for  0*  t  <1+ 

ti>»;n  the  function  <{(*)  of  thu  evalar  argument  *  given  by 

(171  q(»)  -  E  (0*  -  0,  )  •  g  (<T  +  •(©♦  -  0T)  ) 

J»1  J  J  J 


would  have  identical  value*  at  c  *  0  and  r,  *  1 ,  i,  e, 
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(18)  q(°)  ■  <lU), 


Tharafora  for  tomi  valua  of  l  with  0  ^  a  1  wa  have 


«  0 


and  hance 

(19)  Ja  (a)  a  ‘  »*>  (®+  +  k(**  •  ®+))  -  0 


which  would  contradict  (14). 


ON  THE  SIMULTANEOUS  ESTIMATION  OF  A  MISSILE 
TRAJECTORY  AND  ERROR  COMPONENTS  INCLUDING  THE 
ERROR  POWER  SPECTRA  OF  SEVERAL  TRACKING  SYSTEMS 
(Preliminary  Report) 

David  B.  Duncan 
Johns  Hopkins  University 
and 

Pan  American  World  Airwave 
Gutued  Missiles  Range  DlvLslon 
Patrick  Air  Force  Base,  Florida 

1,  INTRODUCTION.  The  problem  urder  consideration  Is  that  of  esti¬ 
mating  the  trajectory  of  a  missile  In  powaied  flight  using  the  data  of  several 
tracking  systems  observing  it  simultaneously.  At  the  same  lime,  It  Is  also 
desired  to  estimate  error  components  prlnolpelly  in  the  tracking  systems  and, 
in  addition,  the  spectre  of  the  autononelated  errors  of  the  observations.  The 
discussion  divides  into  two  parts:  The  development  of  a  Unoar  mixed  model 
up  through  section  4  end  the  development  of  models  with  stationary  prooesses 
In  sections  5  through  8. 

Many  of  the  ideas  discussed  In  the  earlier  part  are  baste  ones  being 
usud  In  the  current  reductions  of  trajectory  data  In  programs  included  under  the 
names  of  'bast  estimets  of  trajectory"  and  "Glotrao".  In  the  second  part, 
similar  ideas  are  extended  to  Include  the  additional  presence  and  estimation 
of  stationary  autocorrelated  errors  In  the  tracking  observations  and,  briefly 
in  section  8,  In  the  trajectory  itself. 

The  early  part  Is  discussed  >ri  the  notation  and  style  of  regression 
theory  and  methods  as  developed  In, .statistical  nooks  of  resent  years  such  es, 
for  example,  Anderson  and  Bancroft  ll  and  CraybiU  [lOj  .  The  lattor  part  In¬ 
volving  spectral  analyses  and  fllt»rr,^  is  developed,  more  than  Is  usual,  In 
he  regre3slon-theory-end-methv ds  style  of  the  first  part.  This  Is  u^nr  'n  the 
hopes  of  better  bringing  out  how  It  all  can  fit  together  with  the  methods  of  the 
first  part  In  complete  unified  anelyseji  Many  references  hp'*e  been  ot  help 
here,  Including  lor  example  Whittle  [isl  and  fUl  ,  Hannan  [llj  ,  Jenkins  ful 
and  Durbin  [9]  ,  no  to  mention  Blackman  and  iiiiiaymto  winch  further  ref-ronces 
ore  niven.  This  brief  list,  of  course,  does  not  even  begin  lo  menticu  allot 
the  many  articles  pertinent  in  this  area. 

No  attempt  is  made  here  to  discuss  many  details  of  methodology  In¬ 
volved  In  wilt  I ng  computing  programs.  These  will  be  written  up  In  reports  by 
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the  author,  E-  E.  McGahcc,  end  3.  B.  Burkett  end  by  L.  B  Collins  end  A. 

Rinaldi  who  have  bean  cooperating  in  the  programming  work.  (As  currently 
planned,  these  reports  will  appear  as  Pan  American  Technical  Staff  Memos  and 
RCA  Data  Reduction  Programming  Memos  respectively.) 

2.  LINEAR-MODEL  WITH  PARAMETERS  All,  HXSP.  A  elmple  first- approach 
model  for  trajectory  data  may  be  willten  as 

(2.1)  v-X/*+., 


where  y  ■  (yj,  ....  yn)'  I?  a  vector  of  n  observations,  X  is  an  nxr  matrix 
of  known  constants,  /4  la  an  rxl  vector  of  fixed  parameters  (regression  cc  • 
efficients)  to  be  estimated,  and  a  la  an  nxl  vector  oi  errors  to  ba  a stl mated 
with,  as  the  name  Implies,  all  zero  expaotallons. 

The  normal  equations  for  estimating  ft  ara 

(2.2)  Ab-g 
or,  moro  fully, 

U'WXjb  -  X'Wy 

whore  the  solution  b  lathe  rxl  vector  estimating  0,  and  W  ■  V’* 
where  V’  U  the  variance  (variance-covariance  matrix)  of  a,  V"var(a). 

Actually,  in  trajectory  applications,  the  problem  Is  Initially  of  a  non¬ 
linear  form,  such  as, 

(2.3)  y  -  f(|3)  +  e 

which  is  handled  Iteratively  In  the  fam'llarform 

(2.3)  y*  -  xfi  +■  e* 

whore  represents  deviations  f)  of  the  parameters  of  Interest  P 

from  Initial  values  similarly  y*  *  ly  -  y0)  where  yQ  ■  I{  and  X 
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la  th«  matrix  of  partial  derivatives  (x^)  ■  (  i^/  d  $  )  of  tha  alamanta  f^ 

of  Up)  with  respect  to  the  elements  of  £ ,  taken  at  P  m  PQ  • 

In  moat  of  what  follows  the  distinction  between  (2.3)  and  (2.1)  la  relatively 
unimportant.  Except  where  distinctions  are  Important  y*  and  will  be 
tuIaUL'u  to  u«  the  obaervatiuus  Uuu  mo  nujoot^i /  param6t#n  raipActlvtly  And 
the  star  notation  wLil  not  be  carried. 

In  a  typical  trajectory  eatlmatlon  problem  the  olementa  of  y,  &  and 
will  be  aubvaotora  end  submatricea.  For  a  ayatem  with  q  obsarviblos  the 
elemonta  yt  of  ywlllba  qxl  aubvaotora  for  anch  aat  of  obaarvattona  at 
eaoh  point  t  in  time,  t  n.  Thua  with  Ml  a  tram,  aay  with  tha  obser- 

vetiona  Pj,  Qj,  P2,  Qj,  R  we  have  q  ■  5;  with  an  Azusa,  aay  with  the  ob- 

servatlona  1,  m,  r  we  hava  q  ■  3,  and  so  on ,  For  a  trajeotory  in  cl  dlman- 
alona,  the  alamanta  ^ t  and  b^  of  0  and  b  will  bn  dxl  vectors,  and  of 

course  in  poworod  missile  flight  d  ■  3.  In  conformance  with  these,  eaoh 
element  xy  of  X  la  a  qxd  submatrix  (usually  of  partial  derivative*) .  In 

passing  ltwtll  be  noted  that  wa  hava  referred  to  the  dimensions  of  y,  X,  0 
and  «  In  counts  of  those  subvector  and  submatrix  elements  and  not  in  terms 
sf  the  ultimate  scalar  element*.  It  will  be  convenient  to  oontlnu*  In  these 
terms  unleaa  otherwise  Indicated. 


Tha  most  popular  forma  of  models  In  current  use  for  tha  reduction  of 
position  data  have  square  diagonal  X  matrices 


(3.4) 


with  as  many  (r  -  n)  trajectory  points  to  bo  ostimated  as  there  are  observations 
(The  aubmattlces  comprising  the  elements  of  course  are  not  necessarily  square 
or  diagonal).  These  we  will  rofar  to  as  stngle-potnt-ln-tlma  models  because 
they  estimate  mathematically  independent  parameters  at  eaoh  point  in  time 
and  emoloy  no  constraints  for  tho  trajectory. 
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I..  the  first-approach  modal  the  error*  are  assumed  unoorralated  over 
time  making  the  variance  and  weight  matrix  diagonal 


■ 

V1 

0 

r 

wi 

0 

(2.5) 

V  - 

v2 

. 

,  w  - 

w2 

0 

v". 

0 

w 

n 

(n  each  of  thane  the  alamort*  are  qxq  submit  rices  which  are  often  assumed 
diagonal. 

In  a  slngle-potnt-ln-tlma  problem  with  time-  unoorrelated  erroru,  the 
normal  aquation*  Ab  •  g  are  alto  diagonal,  being  of  the  form 


“ 

*1 

o  ■ 

V 

*2 

b2 

m 

°2 

l 

» 

0 

L. 

*11 

«n 

. 

In  which  the  solution  for  each  unknown  uau  be  made  separately  In 
at  LV  "  ttn<*  ln  wblob 

at  "  *%'Wtxt(  gt  -  x‘Wyt  ,  »"S . n. 

(2.?)  3x3  3xq  q  x  3  3x1  3xq  qxl 

qxq  qxq 


In  a  coco  with  p  tracking  systems  working  simultaneously,  the  com¬ 
plete  data  can  be  thought  of  as  fitting  the  seme  model  (2.1)  In  the  form, 
with  p  ■  3  say, 
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where  y^,  X^,  ej,  and  correspondingly  Vt  and  are  for  the  ltn  »y!tem 

1  ■  1, . .  .  ,p.  If  the  errori  are  taken  to  be  unoorrelated  between  systems,  the 
normal  equations  are  readily  soen  to  reduce  to  the  same  diagonal  form  (2.6), 
In  which  now, 

•*  '.I,  *'«'  &»'■ 

and  aLt  and  glt  are  the  submatrlcos  and  aubvaotors  defined  as  in  (2.7)  for 
the  Ith  ayatem  at  time  t,  i  ■  1 . .  t  ■  1 . . 


All  of  the  above  are  well-known  and  discussed  In  a  multitude  of  reference!, 
a  typical  example  for  a  sinala  ayatem  being  Davie's  development  of  the  reduc¬ 
tion  of  Clnatheodoltta  data  4J  .  Hare,  It  la  hoped  that  the  review  ha  a  served 
primarily  an  a  relatively  stmpie  Introduction  to  the  notation  and  background 
context  of  work  in  the  following  sections. 


3.  A  LINEAR  MIXED  MODEL.  A  big  weakness  of  the  classic  model  of 
section  2  la  that  it  makes  no  allowances  for  autocorrelation  of  errors  within 
systems,  1.  e.,  time  cnmiiftticna  In  the  errors  at  one  point  in  tha  trajectory  to 
another.  A  bla  step  forward  in  thia  regard  can  be  made  with  the  uae  of  a  mixed 
model  of  the  form 

(3.i)  y«x£>+Zr+e 

where  y,  X,  0  ,  o  ero  aa  already  defined,  and  $  -  ZV  .  or  more  fully, 


thb 
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Isa  qxl  subvsctor  of  error*  In  y  which  can  be  expressed  a*  a  known 
linear  function  zfl  ,  (Jt{  being  qxk)  of  k  error  component*  i  »  (y^, . . . , 

common  to  eli  points  in  time  t  «•  1, . . . ,  n.  For  example,  for  a  given  system 
with  say  q  ■  3  observables  it  might,  be  desired  to  make  ellowanoe  for  a  set  of 
"eero  set"  errors  a^,  a20,  a30  and  a  set  of  linear  drift  coefficients  a^, 

a2T  l,3i •  In  such  an  example  tff  wnniri  be 

r  •».  1 


(3.3) 


lOOt  00 
0  1  0  0  t  0 
0  0  1  0  0  t 


*20 

*30 

•a 

*21 


L-u  J 

Better  model*,  if  available,  are  ones  which  express  the  errors  £t  in  y(  as 
physically  identifiable  error  components  yt  in  the  tracking  system,  such  as, 
til'  firrors,  refraot.on  errors,  etc. 


The  Important  point  that  distinguishes  the  elemento  of  4  from  the  tm* 
Jeotory  parameters  in  (J  is  that  the  y  ‘s  ere  regarded  as  random  variables 
(which  vary  fro-m  flight  to  flight)  with  e  vector  of  means  T,  and  a  variance 
(vari  uhcs-covarience)  matrix  .  (The  error  components  4  and  the  remain¬ 
ing  errors  e  cue  uttuoirelalud . )" 

For  p  system*  working  simultaneously  the  complete  data  lit*  thr> 
aamo  model  (3.1)  in  the 'form,  with  p  ■  3  say, 


-  “1 

X 

ru 

*— • 

O 

o 

_ 1 

—  — 

1 1 

m 

x2 

9* 

0  z2  0 

fa 

’2 

_y3_ 

*3 

0  0  z3 

\ 

e3 

— 

whore,  in  addition  to  lie  terms  already  dleounsud  ior  (2.8),  7<j  and'/j  are 
the  2's  and  %  's  tor  the  Ith  system. 
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In  connsatlon  with  the  general  linear  mixed  model  (3.1)  a  very  useful 
result  can  be  readily  obtained  which  may  be  stated  as  follows:  The  maximum- 
likelihood  estimates  b,  c  of  3  and  tg  ,  (assuming  that  ■y'and  (1  are  Gaussian) 
are  the  same  as  though  we  assume  the  Unoar-wlth-alMlxed  parameters  model 


(3.  5)  y  -  X1  +  X2  t  e 


with  terms  defined  as  In 


yi 

yo 


X 

fi* 

z 

i  + 

°k 

*k 

where  y^  and  -j  are  the  given  nxl  veotors  y  and  <?  ,  and  1^  era 
the  null  and  Identity  matrices  of  order  k,  the  additional  observations  yQ  are 
yu  ■  T  and  contains  the  k  deviation#  '7-  f-  In  other  words  we  can 
treat  i"s  means  (F  as  being  observations  sbout  i  flxsd  ss  mean.  The  aame 
variance  V„  “  V,y  is  taken  for  4Q  as  was  given  for  g  ,  The  full  variance 
end  weiyliL  matrices  for  6  tn  (3.5)  are  thus 


(3,6) 


where 

wo  ■ 


Vj  and 


e-a 

> 

i _ 

o 

,  w  - 

I 

r - 

o 

L  0 

voJ 

o 

woJ 

W j  are  tha  V  and  W  of  the  original  e  In  (3.1)  and 


From  this  pseudu  model  with  psoudo-observatlone  end  all  fixed  parameter!, 
('•i.  in  other  way,)  the  -.atlinetes  H  ■  b^  and  n  ■  b^  are  arven  to  be  given  by 


‘Au 

A1 1 

’b0 

■ 

V 

.  A21 

kn  • 

.b2 

Si. 

with  terms  as  defined  in 
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j-XjWXi 

X'jWX2  " 

rv 

m 

"Xj  W  y  ‘ 

l^wx1 

X2WX2  . 

.  b2- 

.XjWy  . 

which,  In  terms  of  the  given  mixed  model  (3.1)  reduoe  to 


(3 . 7b) 


‘X'.V^X 

.v 


x-WiZ  I  rbi 
Z'WjZ+WqJ  [oj 


‘X1  W  y 
•  •  • 

Z'Wjy+Wpyg. 


it  the  underlying  model  Is  non-llneer  In  the  error  components  2 f ,  the 
matrix  Z  will  consist  of  partial  derivatives,  the  In  the  model  equations 
will  consist  of  Increments  -  /n)  to  be  made  to  Y0,  a  set  of  values 

reached  In  u  previous  Iteration.  The  pseudo-observations  yQ  In  the  equations 
will  consist  of  deviations  yj  ■  (yQ  -  Y^)  m  m  T0).  In  the  first  Iteration 
it  is  natural  to  take  Yn  as?  thus  yj  is  aero  end  the  term  W0y^  drops 

cut  on  the  right  hand  sldu  of  (3.7b).  In  all  subsequent  Iterations  it  will 
appear  aa  W0(^  -  7q). 


In  most  work  a  vector  of  expected  fixed  errors  as  Is  represented  by  %  , 
non  be  oallbreted  out  and  this  Is  usually  done.  Thus  more  often  than  not  the 
effective^  la  0.  Thus  In  a  problem  llnoe1:  In  y  we  have  yQ  ■  0  and 

Wy0  never  appears,  In  the  right  hand  side  of  (3.7b).  In  a  non-linear  prob'nm 
WyJ  v'ill  only  appear  as  before  ofter  the  first  iteration  and  then  as  WQ(y-  7n)  ■ 
-W  Y  .  Details  Ilka  thesa  can  he  reworked  end  oheoked  very  simply  alerting 
from  the  p.seuao- model  (3.5)  or  its  noijftlnear  problem  equivalent. 


Many  trajectory  estimation  pioblems  are  solved  using  the  mlxea  linear 
model  (3.1)  assuming  that  the  verlenoe  metrlces  ■  var  (* )  end  ■  var  (-«') 

are  diagonal.  The  nxn  matrix  A^  and  the  corresponding  nx  I  vector  g, 
are  the  same  as  ths  A  and  g  of  the  standard  model  of  section  3.  (A^  is 
diagonal  with  3x3  submatrlueB  at  salts  nxn  elomonts,  g^  has  "  x  .< 
subvectors).  With  p  systems,  the  elements  of  An  end  g2  from  uaoh  eyetem 
add  together  to  give  an  overall  Aj  j  and  of  tha  eama  nxn  and  nxl 
dimensions. 
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For  a  single  system  A22  “  X'2  WXg  «  Z' Wj  Z  +  W0  is  kxk  and 
symmetric  though  not  diagonal.  a12  "  x'l  WX2  -  X'W^  Is  oorruspontilngiy 
nxk  and  Is  non-null.  A2i“A;2‘  For  p  *y“tsni»,  because  of  the  additional 
error  component  vectors  Tj  entering  with  each  system  A22  1*  kxk  where  now 
P 

k  “  £  kt  anti  h  Is  tha  size  of  y  ,  l  ■  1 ,  p.  A,,  widens  correspond!  no 

1-1  * 

To  take  an  example,  tor  300  seconds  of  data  at  say  10  points  per  second, 
n  -  3,000.  With  p  -  6  systems  say  with  -  60  error  companento  in  all, 

k  -  60 .  Thus  Ajj  would  be  3 , 000  x  3 , 000  and  diagonal  with  3x3  sub* 
matrices  as  elements.  A22  would  be  symmetrio  and  60  x  60  with  scalar 
alemantn.  A,  2  would  be  non-null  3,000  x  60,  with  3x1  aubveators  an 
elements  and  A21  "Ajj, 

Further  dlacuuelons  and  « poll  cations  of  this  model  ura  available  In 
man/ reforonces  including  Brown  |3 ]  ,  Dunoen  5)and|6 1  and  Hendaracn  at 
a!  I2|  .  1  1  l  J  l  J 


StpJJ.  because  A^  above  in  (3  . 7)  Is  diagonal,  It  Is  easily  Invertible; 
large  though  may  ba  Its  slxo.  As  a  result  of  this  It  Is  ocnvunlant  to  aoU.j  fur 
b,  first  in  tha  "roduoed"  normal  equations 

(4 . 1)  I*;*  '  A21  An  A1  ?)  b2  “  *2  *  A21  Aj  J 

and  then  for  bj  by  substituting  b^  In 


Mibl  -  '  Al 


Both  of  these  equations  follow  directly  from  (3.7),  in  this  way  the  largest 
non-diagonal  matrix  to  be  Inverted  Is  the  left  hand  side  one 

(A22  "  A2l  a"A  Ai  l)  ln  whloh  Is  only  kxk, 
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In  this  connection  tt  it  often  useful  to  noto  the  following  development 
of  (4.1)  and  the  subsequent  uss  of  (4.2).  Substituting  for  the  A  terms  from 
(3 .7a)  we  gat 

(4.3)  Of^W1/2DWl/2X2)b;a  -  X’ W1/2DW1/2y 

wnero  D  ■  (l  -  W^X^j  X^W^2)  l«  readily  seen  to  be  Idempotent  and 
nymmetrlc.  from  this  wo  can  write  the  reduced  normal  equations  as 

(4.4)  Xb2  **1' 

M  42  .v  1/2  —4  l/?. 

where  K»  X  X,  g  -  X1  y  and  X  *  DW  '  X2,  y-DWv  y. 

In  this  way  bz  may  be  reqarcjed  as  theM'ftt"  (or  the  result  of  "fitting") 
a  regression  (unweighted)  of  V  on  X  where  y  Is  the  welghtcJ  residual  vector 

(4.4)  "y  n  DWl/2y  «  W^2y  -  a[\1  Wy 

-  Wl/2(y  -  Xjbf)  -  W1/2(y  -  9) 


frotr.  fitting  «  r»g r««nior.  wf  y  <m  ulono  oml  A  comp.iuow  similar  welghtod 
residual  veotois  from  fitting  regressions  of  the  columns  of  X2  on  X^. 

-1  -1 

The  substitution  step  (4.2)  may  be  rewritten  as  bj  ■  ^)1°1  ”  ^ilAl2‘J' 
snr)  then 

(4.S)  bj  ■  bj  -  Kbz 

and  may  be  regarded  as  one  of  "oonxnting"  bf  «  A  1g.  to  give  b,  by  sub» 

-i  1  11  l  l 

iieotlug  the  "corrootlon"  Kb.  «  ,  In  the  sequel  this  will  be  referred 

to  as  the  two-step  reyieuslon  method.,  the  first  step  comprising  the  solution 

giving  'y  and  X  In  which  b*  Is  obtained,  the  second  the  solution  for  b, 

i  *  * 

and  thn  unw  of  this  to  cortoct  bj  to  bj  ■ 
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5.  U| 

A  further  Improved  model  for  trajectory  ssumatlon  would  appear  to  be  the  same 
as  tho  linear  mixed  model  of  section  3,  but  with  allowance  for  stationary  auto¬ 
correlations  among  the  errors  within  systems .  The  linear  terma  Z^  would  take 
oare  of  large  non- stationary  effect*.  An  allowance  for  stationary  autocorrelations 
among  the  subventors  (et  ;  t  *>  1 ,  .  ,  , ,  n)  of  the  errore  e  would  take  oare  of 
a  lot  of  the  remaining  correlations,  Before  passing  on  to  oonsl deration  of  a 
modal  of  this  type  in  the  next  section,  we  will  first  deal  here  with  etatlonary 
autocorrelations  among  tho  trrnrs  In  an  all-flxad  paramateis  modal  of  the  type 
considered  In  section  2, 

In  the  notation 
(5,1)  y  -  x/3  *  e 

Or,  more  fully  for  p  ■  3  systems,  say, 


'1 


i  X 


(5.1a) 


I 


t 


.  y 


i 

x., 


j  / 


\  X 


a  + 


3 


'3  ; 


(of  (3,4)  before),  the  variance  aubmatrloei  In 

V, 


'1 


(5.1b) 


ver  (e)  ■  V 


/ 


arc*  now  of  the  etatlonary  Laurent  form 
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Each  a  la  mint  vlU  li  a  lag-h  outoaovarlanoa  matrix  (or  tha  arrora  at  tlma  t 

in 

and  tlma  t  ♦  h,  t  -  1.  2,  - . .  and  h  "  1,  2,... 

In  getting  almultanaoua  Mttmate*  o(  a  trajaotory  over  icvaral  ayatama 
it  coma*  out  that  a  more  ilmpla  aolutlon  ta  providad  If  we  aaauma  that  appro- 
prletaly  weighted  forma  of  trajectory  aattmatas  obtained  aaparataly  for  aaah 
mvatam  are  tha  obaarvattona  with  the  atatlonery  arrora,  rather  than  tha  original 
ayatam  obaarvattona  thamaalvaa. 

Thua,  for  the  Ith  ayatam  we  may  ohooea  to  taka 


(5 . 2} 


rtt 


V« 


3x1  3x3  3x1 


a*  tno  tth  xlxmant  the  obeervation  vector,  where  l'lt  it,  a  alngla-pelnt- 
In-tlma  oatlmate  (without  aaaymt-.g  atnogorreUttaU  anuia)  of  tha  Individual 
trajai'tory  davlatlona  3*  ■  (  P  t  “  P{0).  1  "  2.  3[  t  ■  l , . . . ,  nj 

ta  a  diagonal  matrix  with  put  equal  to  the  raotprocal  of  tha  atandcrd 
deviation  of  tha  j1*1  element  of  b(t  .  (In  thla  way  dy  la  poattlvu  and 
la  the  jth  diagonal  alemant  of 

•u  “  <xuwltxlt)’1 
for  the  1th  ayatam  at  tlmo  t  In  (2.9J. 


4 


Design  of  Exprirlmonta  673 

Wttn  the  observations  defined  tn  th La  way,  the  matrix  baoomaa 
a  3nx3n  Identity  matrix,  or  In  keeping  with  our  pravloua  notation,  an  nxn 
Identity  matrix  In  which  aaoh  diagonal  element  la  a  3x3  Identity  aubmatrix 
I3,  l  ■  1,  2,  3.  from  thla  the  normal  equation*  for  p  *  3,  aay  ayatama  again,  are 


(5.3)  (Wj  +W2  *W3}b  -  +W2y2  +  W3y3 

where  for  eaoh  ayatem  and  la  of  tho  approximate  laurent  form 


W10 

wti 

W12 

, .  .1 

wll 

W10 

WU 

1  I  • 

(5.4) 

Wl  " 

w12 

i  : 

wu 

wio  1 

1  1 

1 

1  »  1 

We  can  than  write 


(5.5)  b  -  KlVj  +  K2y2  +  X3V3 

whore  K,  ■  (W,  ♦  Vi,  *  le  approximately  Lament,  or,  Inthea'yli 

of  filtering,  1  1  " 

m  m  m 

(5  '•>«)  b,  -  £  k  yl(t+h)+  V  k,.y2(t+h)  +  T  K  y 3 (t  +  h ) 

‘  h--m  lh  h-m  2h  h  «- m  jh 


t  *  1 ,  .  , ,  n 


1 

1 


whore 

(5.6) 


kl  -  (kl(-m) . kl0 . Sr/ 


* 
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U  a  typical  column,  except  lor  aero*  fora  and  alt,  of  K^,  with,  -  klh, 

h  -  1 . m;  l  *  1,  2,  3. 

A  alrnpla  matrlx-and-ragr«»ilon-theory  approach  for  finding  which 

l  a  hdlng  triad  and  v/hich  may  ba  useful  In  axtandlng  to  mora  oomplex  modali 
may  ba  put  at  follow*. 

Suppose  for  tho  moment  that  tha  number  of  Euclidian  coordinate*  la  only 
d  ■  i  inatnad  of  a ,  and  that  tha  element*  w^0,  of  tha  weight  metric* * 

W(  are  thus  aoalar*  Instuad  of  3x3  lubmatrlcaa  „  Aieuma  n  1*  odd,  say,  and 
introduce  a  Fourier  matrix  F(nxn)  ■  Xftj )  oomprlsed  of  the  Fourier  term* 


(S .  7) 


4r 

fif 


i*f  (t-T)O  -D 


wham  F  -  1  -  (n>l)/2,  It  l*  raedlly  aitablUhed  that  this  matrix  1*  orthogonal, 
t.oi  f"1  «  ,  and  that  F  V(  F"1  •PProx  Bj  where  1*  diagonal  with 

"F 

diagonal  element*  comprised  of  tha  mo  it- data  l  led  •ymmetrlc  power  apoctium 

(ilia  symmetric  expected  pw '< «l-igiem)  of  tho  orrora  «i(t  ,  l.o,  e,  *  (a^  , , .  ,  otn>‘ . 

Thu  i  w#  may  writs 

nftp'!  "FJWj  »Wj  ♦wj)“1f"1  rw,  r  1 

'  (FW^  F’1  +  F  WjF'*  ♦  FW1f,"V1rW1F*1  . 


Dili 

-1  -1.1  *PPIU*  “1 
(5.8)  FW  F  1  •  (FV,F  *)  1  -  3[ 

wmen  ws  mall  vmie  a*  i/S^  lu  empiieeuu  tii**  (jouil  mei  tnu  ulauwie  of  dla- 
Uonji  matrices  l*  Ihu  sain*.  as  tlial  of  noalaiB, 


Hones 
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(5.9)  FtF'1  aPPr'*  _ _ 

1/8  +  1/S2  +  1/S3 

where  Is  a  diagonal  matrix  with  diagonal  terms  of  the  form 

l/SjfO/U/Sjtf )  +  l/s2(f)  +  l/s3(f),  f  •  -(n  ♦  !)/?, . 0 . (n  +  l)/2, 

end  from  this 

(5.  JO)  V-F'^F. 

By  assuming  a  certain  degree  of  smoothness  in  each  spectra,  the 

can  be  obtained  as  much  smaller  matrices  with  f  *  -m . 0,  .  . , ,  m,  the 

Tourlur  matrices  can  fca  correspondingly  induced,  end  the  required  filter  elements 
given  by  (5.10)  are  found  by  the  simple  finite  Tourter  transforms  of  the  form 


15.11) 


'lh 


1  £ 


.  i-2Zlhi 

1  v  hi 
e  r 


m 


U 


where  r1(  r.  are  the  diagonal  elements  ot  the  reduced  R  In  whicn 

U-mj  im  i 

>,j  “  (l/SjjJ/d/e^  +  l/*2j  *  1/ S3j)  and  the  s  are  spectral  estimates 
(except  for  a  constant)  from,  say,  a  Blackman-Tukey  (Zl  method,  N  *  2,n  +  1 
■and  the  Imaginary  1  in  the  exponent  of  a  Is  not  to  be  confused  with  the  other 
rr,  for  the  Ith  system.  Because  of  symmetry  (5.11)  reduces  to  the  simple 
cosine  expansions  n*  »hc  trum 


rto  *  2  I  rl)  cos  “jf  hJ 
j**i  N 

In  the  case  of  d»3  dimensions  the  same  approach  extends  tc  th* 
construction  of  three-dimensional  filters .  The  elements  of  F  are  replaced 

by  3  x  3  scalar  matrices  f^I.j  ,  the  diagonal  scalar  elements  s^  of  the 
spectral  matrices  S(  are  replaced  by  3x3  cumplex  spectral  submatrices 
s  c  +  tq(j  and  the  elements  of  the  filters  are  3x3  submatrices 
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fa.  EflTIMATIQ.N SSL JEiiSB .SEEfilBfe- •  Trajectory-frao  ••tlmatus  of  the 
spectra  required  for  the  construction  of  the  filters  In  section  S  oen  be  obtained 
ny  the  oioss-spectrum-of-dlffardnces  method  already  discussed  by  Or.  Wells 
Ln  an  earlier  paper  on  this  program,  (Duncan  and  Wells  ["8]  ).  Once  a  process 
of  trajectory  estimates  obtained  by  the  method  of  aeotton  5  La  available* 
the  Initial  spuotral  estimates  can  ba  Improved  by  Iterations  of  the  semi  form 
of  procedure. 

To  explain  this,  the  method  beaed  on  differences  can  be  Identically 
re- represented  as  a  method  baaed  on  rastduala  jj.  ■  y.t  *  7*  (  , 

t  -  1,  2,  3  where  I  ) 


(cf .  Dunoan  and  Carroll  [?]  ) .  The  desired  apeotral  estimates  '■i (£) ,  '*2(f), 


^j(f)  can  be  obtained 
normal  aquations. 

(In  the  car, e  of 

d  •  1 

dimension) 

hy  solving  tha 

•ll*l<f> 

♦  *12*2^ 

+ 

•13«3(f) 

-  V* 

(6.1)  «2 1  •  ltt) 

+  a2Zs2(f) 

f 

a23*3^ 

-  «ua« 

a31Sl^ 

+ 

+ 

*33*4® 

*  ;U3® 

where  the  right-hand  side  values  are,  say,  estimates  of  the  spectre 

s  (0,  s  (f)  of  the  residual  processes,  and  where  * 

-  Z  u3 


The  solution  Is  carried  out  at  each  frequency  f. 
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S77 


jb^  of  the  trajectory  has  been  made  with  filters 
Kl>  K^,  Kj  the  residuals  oan  now  be  recomputed  as  fult"yit-\}‘  Th0 
equations  (6.1)  are  then  solved  again  with 


(6. ?) 


<  V 


1J 


.i 

r2.l  "  V 


ncre 


‘u 


Is  the  *th  frequency  element  of  R,  *  FK|  P"1  Steps  like  these 


mtarsparsed  with  the  iterations  for  getting 


w 


ultlmatnly  yield  good  estimates 

of  hoth  the  spectre  and  the  trafect.iry.  in  the  oasa  of  d  ■  3  dimensions,  thu 
same  approach  extends  to  the  estimation  of  3  x3  complex  spectra)  matrices. 
(Note:  In  the  discussions  of  neotlons  5  and  C,  )  b*(  sometimes  refers  to  devla- 

If 


tlons  b*  »  b  -  as  previously  Indlualwd,  and  sometimes 

this  la  kept  in  mind,  no  trouble  should  arise  In  distinguishing  which  Is  which.) 


to[bt} 


7.  UNEAR-MIXEP  MfeaSk^imMiaCPMElAITO  BBBQM  ■ 
Ideas  oan  be  adapted  to  the  mixed  model 


The  same 


(7.1) 


X0  ♦  ZT  *  € 


In  which  the  errors  £  are  autooorralsted  as  In  section  S. 

By  defining  the  nh$s.'v„itons  as  In  section  5  th e  system:  aubmatiues 
Xj  of  X  can  »».oh  be  reduced  to  nxn  (or  CmxSx)  Identity  matrices .  If  the 

model  for  the  system  obaervattons  Is  ltnsar  In  ,  the  modified  mode1  will  't 
nr"' -linear  in  v  .  The  vector  V  will  thus  represent  deviations  f*  m  ('4t  -  V 
at»i  Z  will  consist  of  partial  derivatives. 

Rewriting  the  model  In  the  psoudo  all-ftxad  parameters  form 


(7.2) 


*1  Pi  +  x2  ^2  +  £ 


which,  In  more  detail  represents 


* 


fa* 
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with  terms  defined  ta  before  (cf  (3 .5)),  the  first  steps,  talking  In  terms  oi 
the  two-step  regression  approach  (section  (4)),  la  to  fit  regressions  of  y  on 
Xj  and  of  Xz  (l.e.,  of  the  columns  of  Xj)  on  forming  the  residual! 

(7.3)  y  -  DWi//2y  and  X*  -  DW1/2X2 


where 


D  -  1  -  W1/2  Xt  (Xj  WXj)' 1  X'j  W1/2 

This  will  reduce  to  slmpln  filtering  operations  on  y  and  on  Z  ths  details  of 
which  can  be  readily  developed  jby  the  method  of  section  5.  In  this  step  a 
first-step  trajectory  estimate  bj  Is  also  obtained  by  filtering. 

In  the  second  step  ths  equations 


(7  4)  X  Xb2  -  Xsr' 

era  solvod  for  bj  and  b*  is  corrected  using  a. 

<?•  5)  j»4t  "  Vit  ■  blt  “  altbZtj  '  l-1'  3l 

The  desired  estimates  of  the  reapeotlve  error  proosssss  could  then  be  stained 
by  solving  equations  of  the  form  (6.1)  where  tho  right  hand  side  terms  are 
based  on  these  residuals.  This  would  Involve  Ignoring  the  effect  In  the  re¬ 
siduals  of  slt(b2t  -  p2t)  which  would  not  cause  any  problem  alnoe  b2t  to 

fitted  over  the  whole  time  span  of  date  end  bgt  -  /?2t  would  have  a  relatively 
low  varlanoe.  To  start  the  process  the  initial  trajectory  ostimnta«  In  the  first 
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two-step  solution  would  be  obtained  using  filters  from  previous  experience,  or 
at  the  very  worst,  using  Identity  filters. 

The  estimation  procedure,  in  summary,  would  consist  of  several  three- 
step  Iterations,  two  giving  closer  estlmetes  of  the  trajeotory  parameter*  In  ft 
and  the  error  components  In  ^  ,  the  third  giving  olotei  estimates  of  tho  error 

spectra, 


(1)  In  connection  with  the  spectral  estimates,  the  emphasis  here  has 
been  placed  on  the  role  these  play  In  leading  to  better  trajectory  estimates.. 

The  error  spectra  however,  and  especially  those  of  the  Initial  tracking  system 
observations ,  are  of  considerable  Interest  In  themsslves.  With  title  In  mind, 
when  a  set  of  estimates  In  the  trajeotory  coordinate  system  have  converged, 

It  will  then  be  desirable  to  project  the  trajectory  estimates  Into  the  coordinates 
of  each  ol  the  tracking  systems.  The  residuals  of  the  Initial  observations  from 
these  wul  then  provide  a  basis  for  gatting  good  sstlmatss  of  the  speotra  of  the 
anors  in  the  Initial  observations.  Euoh  will  consist  of  a  linear  combination  of 
the  spectral  estimate  of  the  residual  Concerned  and  the  spectral  estimate  of  the 
error  In  the  trajectory  estimate. 

In  the  event  that  the  variance  matrix  of  the  Initial  observations  turns 
out  at  this  point  to  be  much  different  from  that  Initially  assumed  in  transforming 
to  the  trajectory  coordinates,  It  msy  be  considered  desirable  to  make  another 
Iteration  starting  with  the  latest  waighto. 

(2)  A  direction  ?«r  Improvement  Is  that  of  allowing  for  spectra  t:  d 
filters  which  ran  change  with,  some  flexibility  over  time.  It  la  hoped  to  d  ivelop 
methods  for  handling  this  along  wkh  ihe  development  of  a  promising  mode'  i  • 
which  we  now  rofer  very  briefly. 

(3)  Perhaps  the  most  interesting  end  usoful  features  of  the  foregoing 
mixed- model-plus-flltertng  methods  are  their  potent!*!  usefulness  in  the 
analysts  of  data  In  a  more  advanced  model.  This  model  Is  the  same  a*  tho 
linear  mixed  model  with  stationary  errors  In  seotlon  7  except  that  the  specllJ- 
catton  for  the  trajectory  Itself  Is  no  longer  of  the  slngle-polnt-in-tlme  form 
(with  a  diagonal  X  matrix  for  each  system).  The  trajectory  Is  now  represented 
at  the  centers  of  progrsBBtve  medium- length  intervals  by  a  low  degraa  polynomial 
(say  a  quadratic)  plus  a  deviation  $  ,  t  ■  1, .. . ,  n  for  suooesslve  Intervals 

are  assumed  to  form  a  stationary  time  series. 


4 
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Similar  step-bystep-regrasslon-plui-flltortng  method*  can  be  used  to 
give  estimates  of  (l)  the  polynomial  componant  and  (11)  the  stationary  oom- 
ptmeni  and  thus  the  combined  components  of  the  trajectory,  (111)  the  error  com¬ 
ponents  In  the  tracking  system*,  (Iv)  the  spectra  of  the  error  processes  and 
(v)  the  spectrum  of  the  stationary  trajectory  components.  A  modal  with  poly¬ 
nomial  plus  stationary  components  in  a  signal  Is  suggattad  in  similar  contexts 
by  several  writers,  e.g. ,  Zadeh  and  Ragazztnl  [1  7),  and  Ilunzlgei  ^3]  . 

Ihe  prospect  here  of  also  simultaneously  estimating  the  spectrum  of  the 
stationary  trajectory  componant  has  special  appeal.  Roughly  speaking,  the 
traleuiory  dale,  In  this  approach,  will  be  fitted  under  constraints  as  tight  as 
the  data  themselves  will  condone.  The  estimates  will  have  an  appreciable 
part  ol  the  low- variance  advantages  that  can  be  added  by  smoothing  without 
having  lost  mucn  oi  the  responsiveness  which  Is  shown  by  the  data  to  bo  needed. 
Current  work  lb  in  progress  on  these  aspects. 
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